| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
| Name | Name | Last commit date | ||
|---|---|---|---|---|
_________ ____ ___ ___ / /__ __ / ___/ __ \/ __ `__ \/ _ \/ __/ |/_/ / /__/ /_/ / / / / / / __/ /__> < \___/\____/_/ /_/ /_/\___/\__/_/|_|
Open source extensions for the Comet SDK.
These extensions are created and supported by the community and are not an official project of Comet ML. We welcome contributions!
pip install cometx --upgrade
To use these command-line functions, you can set your Comet API key and URL override using command-line flags:
cometx --api-key="YOUR-COMET-API-KEY" COMMAND
If you are a Comet on-prem user, and your installation does not use smart-keys, you'll also need to set the URL override:
cometx --api-key="YOUR-COMET-API-KEY" --url-override="https://your-companys-comet.com/clientlib/" COMMAND
cometx is composed of a series of commands that are useful independently, and can be used together to create sophisticated tools for ML management.
For all commands, use the --help flag to get additional information.
These flags are availble before a command:
This command shows the cometx version:
This command can be used globally, or for individual commands:
This command is used to:
cometx list examples:
cometx list WORKSPACE/PROJECT/EXPERIMENT-KEY-OR-NAME cometx list WORKSPACE/PROJECT cometx list WORKSPACE cometx list
For more information, cometx list --help
This command is used to:
cometx count examples:
cometx count cometx count --workspaces-only cometx count --with-projects cometx count --with-experiments cometx count --count-all cometx count --limit 10
For more information, cometx count --help
This command is used to:
Note: For detailed information on copying experiments from one Comet installation to another, see MIGRATIONS.md.
cometx download examples:
cometx download WORKSPACE/PROJECT/EXPERIMENT-KEY-OR-NAME [RESOURCE ...] [FLAGS ...] cometx download WORKSPACE/PROJECT [RESOURCE ...] [FLAGS ...] cometx download WORKSPACE [RESOURCE ...] [FLAGS ...] cometx download [RESOURCE ...] [FLAGS ...]
Where [RESOURCE ...] is zero or more of the following names:
If no RESOURCE is given it will download all of them.
To download artifacts:
cometx download WORKSPACE/artifacts/NAME [FLAGS ...] cometx download WORKSPACE/artifacts/NAME/VERSION-OR-ALIAS [FLAGS ...]
To download models from the model registry:
cometx download WORKSPACE/model-registry/NAME [FLAGS ...] cometx download WORKSPACE/model-registry/NAME/VERSION-OR-STAGE [FLAGS ...]
To download panels:
cometx download WORKSPACE/panels/NAME-OR-ID [FLAGS ...] cometx download WORKSPACE/panels [FLAGS ...]
For more information, cometx download --help
This command is used to:
Note: For detailed information on copying experiments from one Comet installation to another, see MIGRATIONS.md.
cometx copy examples:
cometx copy SOURCE DESTINATION cometx copy --symlink SOURCE DESTINATION cometx copy --path /base/path SOURCE DESTINATION cometx copy --path ~/Downloads SOURCE DESTINATION
where SOURCE is:
where DESTINATION is:
Not all combinations are possible:
| Destination → Source ↓ |
WORKSPACE | WORKSPACE/PROJECT |
|---|---|---|
| WORKSPACE/*/* | Copies all projects | N/A |
| WORKSPACE/PROJ/* | N/A | Copies all experiments |
| WORKSPACE/PROJ/EXP | N/A | Copies experiment |
The --path option allows you to specify a base directory where your workspace folders are located. This is useful when your downloaded experiments are stored in a specific directory structure.
Examples:
# Copy from experiments in /data/experiments/workspace
cometx copy --path /data/experiments workspace dest-workspace
# Copy from experiments in your home directory
cometx copy --path ~ workspace dest-workspace
# Copy from experiments in Downloads folder
cometx copy --path ~/Downloads workspace dest-workspaceFor more information, cometx copy --help
This command is used to log a resource (metrics, parameters, asset, etc) file to a specific experiment or experiments.
cometx log examples:
cometx log WORKSPACE/PROJECT/EXPERIMENT-KEY FILENAME ... --type=TYPE cometx log WORKSPACE PANEL-ZIP-FILENAME ... --type=panel cometx log WORKSPACE PANEL.py ... --type=panel cometx log WORKSPACE PANEL-URL ... --type=panel cometx log WORKSPACE/PROJECT --type=other --set "key:value" cometx log WORKSPACE --type=other --set "key:value"
Where TYPE is one of the following names:
For more information, cometx log --help
This command migrates users into workspaces from a source Comet environment to a destination environment. It reads workspace membership from a chargeback report fetched from the source environment's admin API (or a pre-downloaded local file) and invites each user to the corresponding workspace in the destination environment by email. If a user with that email does not yet exist in the destination, they will be provisioned access once they sign up.
cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY [FLAGS ...] cometx migrate-users --api-key DEST_KEY --chargeback-report /path/to/report.json [FLAGS ...]
Arguments:
# Dry run — preview what would happen
cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY --dry-run
# Execute the migration
cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY
# Use a local chargeback report file
cometx migrate-users --api-key DEST_KEY --chargeback-report /tmp/chargeback_reports.json
# Self-hosted environments — provide explicit URLs
cometx migrate-users --api-key DEST_KEY --url https://comet.dest.example.com \
--source-api-key SOURCE_KEY --source-url https://comet.src.example.comFor more information, cometx migrate-users --help
This command is used to rename duplicate experiments within projects. When multiple experiments share the same name in a project, this command renames the duplicates to NAME-1, NAME-2, etc. while avoiding conflicts with existing names.
cometx rename-duplicates [PATH] [--dry-run] [--debug]
cometx rename-duplicates examples:
cometx rename-duplicates # Process all workspaces cometx rename-duplicates WORKSPACE # Process all projects in a workspace cometx rename-duplicates WORKSPACE/PROJECT # Process a single project cometx rename-duplicates WORKSPACE/PROJECT --dry-run # Preview changes without renaming
Where PATH is optional and can be:
For more information, cometx rename-duplicates --help
To delete experiments assets:
cometx delete-assets WORKSPACE/PROJECT --type=image cometx delete-assets WORKSPACE/PROJECT/EXPERIMENT --type=all
Type can be valid asset type, including:
For more information, cometx delete-assets --help
To enable auto-logging of your notebooks in Jupyter environments:
cometx config --auto-log-notebook yesTo turn auto-logging of notebooks off, use:
cometx config --auto-log-notebook noIf you keep the generated experiment URLs in the notebook, but later edit the notebook, the notebooks will be updated in all of the experiments created in the notebook.
For more information, cometx config --help
cometx reproduce [-h] [--run] [--executable EXECUTABLE] COMET_PATH OUTPUT_DIR
This command is used to reproduce experiments by copying files to a specified output directory.
For more information, cometx reproduce --help
cometx update [-h] [--debug] COMET_SOURCE COMET_DESTINATION
To update existing experiments.
cometx update SOURCE DESTINATION
where SOURCE is a folder:
where DESTINATION is a Comet:
For more information, cometx update --help
cometx admin [-h] [--host HOST] [--debug] ACTION [ARGUMENTS ...]
To perform admin functions
Generate a chargeback report from the Comet server.
cometx admin chargeback-report [YEAR-MONTH]
Arguments:
Output:
Examples:
cometx admin chargeback-report cometx admin chargeback-report 2024-09
Generate a usage report with experiment counts and statistics for one or more workspaces/projects.
cometx admin usage-report WORKSPACE [WORKSPACE ...] cometx admin usage-report WORKSPACE/PROJECT [WORKSPACE/PROJECT ...] cometx admin usage-report --app
Arguments:
Options:
Output:
Examples:
# Generate report for a single workspace cometx admin usage-report my-workspace # Generate report for multiple projects cometx admin usage-report my-workspace/project1 my-workspace/project2 # Generate report with weekly grouping cometx admin usage-report workspace1 workspace2 --units week # Generate report with daily grouping, don't auto-open cometx admin usage-report workspace --units day --no-open # Launch interactive web app cometx admin usage-report --app
Generate an organization growth & adoption report as a single self-contained HTML page, built entirely from the admin chargeback report. Distinct from usage-report (experiment-count PDF): growth-report gives an org-wide view of workspaces, users, and platform adoption, per workspace/department.
Requires an admin API key — the report is derived entirely from the admin chargeback endpoint, and with a non-admin key the command errors out (non-zero exit).
cometx admin growth-report [WORKSPACE ...]
Arguments:
Options:
Two time concepts:
Growth rates: the growth KPIs are computed from chargeback createdAt timestamps as new_in_window / count_before_window × 100. Workspace creation is proxied from each workspace's earliest member createdAt.
Report sections: an Organization overview (chargeback) section (org-wide KPIs, workspace platform mix, total-vs-active and added-vs-deleted charts, and a by-workspace table), a Users section (Total / Active / Active % / New-in-window KPIs plus over-time charts), a Leaderboards section (top-N and active-aware bottom-N workspaces by experiments/projects and users by Opik spans / EM activity, exact from chargeback), and a Personal vs Service accounts split (service accounts from the admin API, with a labeled regex fallback). Each people section degrades independently: if its inputs are missing, a warning is printed and only that section is dropped.
Caveats:
Examples:
# Org-wide report, default 7-day window cometx admin growth-report # 30-day window, scoped to two workspaces cometx admin growth-report --window 30d my-workspace another-workspace # Write to a file without auto-opening cometx admin growth-report --no-open --output growth.html # 30-day activity window and top/bottom-10 leaderboards, excluding personal # workspaces named like "user-..." cometx admin growth-report --active-window 30d --leaderboard-top-n 10 \ --exclude-personal --personal-pattern '^user-'
Generate a GPU usage report for one or more workspaces/projects with detailed GPU metrics analysis.
cometx admin gpu-report WORKSPACE [WORKSPACE ...] --start-date DATE cometx admin gpu-report WORKSPACE/PROJECT [WORKSPACE/PROJECT ...] --start-date DATE
Arguments:
Options:
Output:
Examples:
# Generate report for a single workspace from a start date cometx admin gpu-report my-workspace --start-date 2024-01-01 # Generate report with date range cometx admin gpu-report my-workspace --start-date 2024-01-01 --end-date 2024-12-31 # Generate report for multiple projects cometx admin gpu-report workspace1/project1 workspace2 --start-date 2024-01-01 # Generate report with custom metrics cometx admin gpu-report my-workspace --start-date 2024-01-01 --metrics sys.gpu.0.gpu_utilization sys.gpu.0.memory_utilization # Generate report and automatically open PDF cometx admin gpu-report my-workspace --start-date 2024-01-01 --open
For more information, cometx admin --help
Note: For detailed information on admin commands, see README-ADMIN.md.
cometx smoke-test [-h] [--exclude [EXCLUDE ...]] [--debug DEBUG] COMET_PATH [include ...]
Perform a smoke test on a Comet installation. Logs results to WORKSPACE/smoke-tests or WORKSPACE/PROJECT.
Examples:
Run all tests:
cometx smoke-test WORKSPACE # project defaults to smoke-tests cometx smoke-test WORKSPACE/PROJECT
Run everything except mpm tests:
cometx smoke-test WORKSPACE/PROJECT --exclude mpm
Run just optimizer tests:
cometx smoke-test WORKSPACE/PROJECT optimizer
Run just metric tests:
cometx smoke-test WORKSPACE/PROJECT metric
Items to include or exclude:
For more information, cometx smoke-test --help
In this section we'll explore some common scenarios.
A useful idiom is to set your Comet environment variables on the line of a command. In this manner, you can set the COMET_URL_OVERRIDE and COMET_API_KEY for different installations.
Of course, you don't have to set the environment variables if you are copying experiments on the same Comet installation.
Here is how you one could download the experiments in WORKSPACE/PROJECT from http://comet.a.com:
cometx --api-key=A-KEY download WORKSPACE/PROJECTThe cometx download command downloads all of the Comet experiment data into local files. Note that WORKSPACE/PROJECT refers to a workspace and project on http://comet.a.com.
One could then copy the downloaded experiment data with a similar command:
cometx --api-key=B-KEY copy WORKSPACE/PROJECT NEW-WORKSPACE/NEW-PROJECTNote that WORKSPACE/PROJECT now refers to a directory, and NEW-WORKSPACE/NEW-PROJECT refers to a workspace and project on http://comet.b.com.
Similarly, one can copy all of the projects by first downloading them:
cometx --api-key=A-KEY download WORKSPACEand then copying them:
cometx --api-key=B-KEY copy WORKSPACE NEW-WORKSPACESimilarly, one can copy a single experiment first downloading it:
cometx --api-key=A-KEY download WORKSPACE/PROJECT/EXPERIMENT-NAME-OR-IDand then copying it:
cometx --api-key=B-KEY copy WORKSPACE/PROJECT/EXPERIMENT-NAME-OR-ID NEW-WORKSPACE/NEW-PROJECTWARNING: Running the tests will create experiments, models, assets, etc. in your default workspace if not set otherwise.
To run the tests, you can either export all of these items in the environment:
$ export COMET_USER="<USERNAME>"
$ export COMET_WORKSPACE="<WORKSPACE>"
$ export COMET_API_KEY="<API-KEY>"
$ pytest testsOr, define workspace and api_key in your ~/.comet.config file:
$ export COMET_USER="<USERNAME>"
$ pytest tests| Back | FazBrowse Home | New Git URL |