Milk Parlor / the gateway
Sends your model calls and returns the answers. Saves eligible request–response pairs in the background.
MILK INFRASTRUCTURE
THE MILK PROGRAMMODEL CALLS HAVE A SECOND ACT.
Point
OpenAI
at Milk.
Your app gets its answer.
Milk puts the saved traffic to work.
Same SDK. Same calls. Your models.
REQUEST + RESPONSE
OPENSOURCE
CODE
CODE
MEET YOUR
SECOND ACT.+SUMMARIES
MODEL EXPERIMENTS
SECOND ACT.+SUMMARIES
MODEL EXPERIMENTS
DON’T JUST CALL A MODEL.Learn from what goes through it.
THE TWO-LINE SETUP
Keep the SDK. Add Milk.
Keep the official OpenAI SDK and your existing calls. Change the URL and key.
# connect the OpenAI SDK to Milk Parlor
+OPENAI_BASE_URL=https://parlor.milkinfrastructure.com/v1
+OPENAI_API_KEY=<your Milk key>
# Responses and Chat Completions stay unchanged
your app→Milk Parlor→your model
└─ selected completed request + answer→object store→Milk Man / model tests + training
AFTER THE ANSWER
But wait. There’s data.
Milk Man / the agent
Give it a goal. It runs commands, starts cloud models, measures results, and saves what it learns.
Parlor handles the request. Milk Man works on what comes after.
01 / count
Let the data arrive.
The heartbeat counts saved exchanges. No model call.
02 / summarize
Reach a milestone.
A job reads the new data and the previous summary.
03 / read
Keep the result.
The saved file fills the dashboard. Open it to see what changed.
1001,00010,000100,000
Example milestones · set with MILK_SUMMARY_THRESHOLDS. One exchange is a request and its response.
Then, put the data to work.
01collect
Save each eligible request and answer under your account's unique ID. Related exchanges stay grouped as one task.
02understand
Count new exchanges. At configured milestones, save a summary of topics, tasks, and response times.
03make examples
A teacher model writes new tasks inspired by the traffic. Keep training tasks separate from the tasks used to judge improvement.
04train + compare
Train an open model, tune how it runs, and compare answers, speed, and cost on the same tasks. Qwen3.5-0.8B is the first training base.
05review route
Prepare a gateway rule for the selected model. The operator reviews and signs it before customer traffic moves.
Give Milk Man a task: run a model, compare its settings, or train from saved data. It runs the jobs and keeps their results. Closing the dashboard does not stop it.
Where do the models and data live?
Environment variables select local files or S3-compatible storage, managed inference APIs, and owned Baseten or Modal GPUs. Model files are cached separately from the runtime image. Milk Man can run locally while the data and compute stay in the cloud.
Does this guarantee a better model?
No. Milk keeps the experiment and its result, including failures. Speed improvements and better answers are different measurements. The goal is to beat a chosen baseline on your tasks, not to claim a universal ranking.
THE DEMONSTRATION
Actual screens. Actual work.
Recorded development runs, not live status. Click a screen to see it full size.
connect with the SDK
follow the next summary milestone
