syncer.py 2.1 KB

12345678910111213141516171819202122232425262728293031323334353637383940414243444546
  1. import logging
  2. from dataclasses import dataclass
  3. from ray.train._internal.syncer import SyncConfig as TrainSyncConfig
  4. from ray.util.annotations import PublicAPI
  5. logger = logging.getLogger(__name__)
  6. @PublicAPI(stability="beta")
  7. @dataclass
  8. class SyncConfig(TrainSyncConfig):
  9. """Configuration object for Tune file syncing to `RunConfig(storage_path)`.
  10. In Ray Tune, here is where syncing (mainly uploading) happens:
  11. The experiment driver (on the head node) syncs the experiment directory to storage
  12. (which includes experiment state such as searcher state, the list of trials
  13. and their statuses, and trial metadata).
  14. It's also possible to sync artifacts from the trial directory to storage
  15. by setting `sync_artifacts=True`.
  16. For a Ray Tune run with many trials, each trial will upload its trial directory
  17. to storage, which includes arbitrary files that you dumped during the run.
  18. Args:
  19. sync_period: Minimum time in seconds to wait between two sync operations.
  20. A smaller ``sync_period`` will have the data in storage updated more often
  21. but introduces more syncing overhead. Defaults to 5 minutes.
  22. sync_timeout: Maximum time in seconds to wait for a sync process
  23. to finish running. A sync operation will run for at most this long
  24. before raising a `TimeoutError`. Defaults to 30 minutes.
  25. sync_artifacts: [Beta] Whether or not to sync artifacts that are saved to the
  26. trial directory (accessed via `ray.tune.get_context().get_trial_dir()`)
  27. to the persistent storage configured via `tune.RunConfig(storage_path)`.
  28. The trial or remote worker will try to launch an artifact syncing
  29. operation every time `tune.report` happens, subject to `sync_period`
  30. and `sync_artifacts_on_checkpoint`.
  31. Defaults to False -- no artifacts are persisted by default.
  32. sync_artifacts_on_checkpoint: If True, trial/worker artifacts are
  33. forcefully synced on every reported checkpoint.
  34. This only has an effect if `sync_artifacts` is True.
  35. Defaults to True.
  36. """
  37. pass