DEV Community

Ti Gg
Ti Gg

Posted on Edited on

Stop Leaving PyCharm to Check a Training Run

A training job finishes. The interesting part starts: which run was that, what were its
parameters, and did that loss curve actually go down? For most ML teams the answer lives in three
browser tabs — the MLflow UI on port 5000, a W&B project, an Optuna dashboard — and every check
costs a context switch, a re-authentication, and a new set of tabs to lose.

ML Experiment Cockpit is a free
JetBrains plugin that puts a read-only view of those servers inside a tool window in PyCharm,
IntelliJ IDEA, WebStorm, GoLand or CLion. It does one thing: it shows you what your tracking
servers already contain, without leaving the editor.

What it shows you

Four tabs, one per kind of server:

  • Experiments — your MLflow experiment tree (experiments, runs, status, parameters). Pick a run, pick a metric, and the plugin charts its history: the curve, the last value, and the step the value was recorded at.
  • Optuna — studies and trials read from an Optuna Dashboard.
  • Models — the MLflow Model Registry (registered models and their versions).
  • Sweeps — W&B sweeps, read-only, with your own W&B API key.

The last three tabs stay empty until you configure them, so a plain MLflow setup only ever sees
the first tab.

Getting started

  1. Open the tool window: View | Tool Windows | ML Experiment Cockpit.
  2. Set your server in Settings | Tools | ML Experiment Cockpit: the MLflow tracking URI, for example http://localhost:5000. Leave the other fields empty if you do not use Optuna or W&B.
  3. Press Refresh in the tool window toolbar.

That is the whole setup. There is no account, no key to obtain for MLflow, and no service of ours
in the middle.

The code you would chart

The plugin never runs your training code and never writes to your tracking server — it reads what
the server already has, the same way the MLflow SDK does. So the code stays exactly where it
belongs, in your project:

import mlflow

mlflow.set_tracking_uri("http://localhost:5000")

with mlflow.start_run(run_name="exp-42"):
    mlflow.log_param("lr", 3e-4)
    mlflow.log_param("batch_size", 32)
    for step in range(200):
        mlflow.log_metric("loss", 1.0 / (step + 1), step=step)
Enter fullscreen mode Exit fullscreen mode

Start that, then refresh the tool window: exp-42 appears under your experiment with lr and
batch_size in the parameters panel, and the loss metric is one click away as a curve. If the
run looks wrong, you know which parameter to change before you queue the next one.

The limits, so you do not have to discover them

This is a viewer, not a second MLflow client, and it has boundaries on purpose:

  • up to 500 experiments, 100 runs per experiment, 500 registered models, 500 model versions, and 5 000 metric points per run;
  • W&B sweeps are read without pagination — you see the first 50 sweeps;
  • it is read-only: it will not start a run, change a parameter, tag a model or delete anything;
  • it renders what your server returns within those limits, so if a server is configured to hide something, the plugin cannot show it.

If you need to write to MLflow from a script, keep using the MLflow SDK — that is still the
right tool.

What happens to your data

The plugin talks to the servers you configure — the MLflow tracking server, the Optuna
dashboard, the W&B endpoint — and to nothing else. There is no telemetry, no analytics, no
advertising, and no code that sends anything to JetBrains or to us. We do not operate a proxy: the
requests go straight from your IDE to your server.

Your server URLs and W&B entity/project are stored in the IDE's own configuration directory, in
mlExperimentCockpit.xml, and are not synchronized through IDE Settings Sync. Your W&B API key
lives in the IDE's credential store (PasswordSafe), never in a settings file. If you uninstall
the plugin, the mlExperimentCockpit.xml file stays behind in the IDE's configuration
directory — your MLflow, Optuna and W&B data stays on your servers either way.

Who should skip it

If your workflow is "one notebook, one local run, no tracking server", a plugin is overkill. If you
need to launch training jobs from the IDE, this is not that tool either. If you do keep an MLflow,
Optuna or W&B server, and you keep coming back to it to check a curve, this is the shortcut.

Try it

If something does not work, write to [email protected] with your IDE and its build
number (Help | About), the plugin version, the steps to reproduce the problem, the exact message
shown in the tool window, and the version of your server. Please do not send credentials or
confidential experiment data in a support request.

One caveat worth stating plainly: JetBrains records your download against the marketplace listing,
and anything you post as a comment or a rating is public on that page. That is JetBrains' own data
flow, not ours.

Supporting the maintenance (optional)

The plugin you install is free, and the four tabs above stay free for everyone. What costs me
time is keeping it alive: reading the bug reports, testing against new IDE builds, and keeping up
with the MLflow API changes. If the plugin saved you a few context switches this week and you
want to say thanks, you can leave a small tip here:
https://tggster.gumroad.com/l/piazge (CHF 3+, whatever you want to give).

To be precise about what that is: it is voluntary support, not a purchase. It does not unlock
any feature, it does not include support, and the plugin works exactly the same without it. A paid
Pro tier with an extra Compare view is now available on the JetBrains Marketplace (with a
free trial); it is sold there, never through this tip page.


Disclosure: this article was written with the help of AI and reviewed before publication.

Top comments (0)