Bring a model.
Start with the model you want to host and the hardware available to run it. Establish a memory budget and the workload you need to serve.
Your model. Your machine.
Local hosting. Automatic tuning.
Local models.
Tuned to
your machine.
Host models on your own hardware. strided is being built to tune kernels, model settings, and memory use together as your workload changes, so you spend less time tuning and more time using your models.
In development. Grounded in real workloads.
stridedCoordinates the stack
Design direction- WorkloadRequests · context
- ModelHyperparameters
- RuntimeKernels · execution
- MemoryCache · layout
- HardwareGPU · CPU · RAM
One control loop, across the stack.
strided is being built to coordinate kernel choices, hyperparameters, memory, and scheduling, then verify their combined effect on the workload.
Spans Workload · Model · Runtime · Memory · Hardware
Select an adjustment to see which parts work together.
The system
A model host that
knows its limits.
Local hosting brings the model, machine, and workload together. The settings need to make sense for all three.
A longer prompt needs more memory. More requests compete for the same resources. Yesterday's settings may not fit today's work.
strided is designed to host local models and adjust how they run. It will use available resources and live workload signals to coordinate kernel choices, hyperparameters, and memory settings within the limits you set.
- Your model
- The weights, context length, and memory it needs.
- Your hardware
- The GPU, CPU, and memory available to run it.
- Your workload
- Interactive replies, longer prompts, or work waiting in a queue.
The goal: useful models on your hardware, with less manual tuning.
Explore the visionVision
From local models
to data center systems.
Local hosting is our starting point.
We're building automatic runtime tuning for models on your own hardware. Our longer-term ambition is to extend that control to model serving across shared compute and, eventually, the data center.
-
Current focus · In development
Local model hosting
Host models on one machine and automatically adjust memory, batching, and concurrency. Start by making that control loop useful, observable, and reliable.
-
Expansion path
Shared model serving
Extend the foundation to teams running multiple models across multiple machines. Coordinate workload placement, request queues, and capacity across shared compute.
-
Long-term ambition
Data center systems
Connect model serving and runtime decisions to scheduling, hardware, networking, power, and thermal constraints. Work toward coordinated control of the full system.
The strided control loop carries forward: observe the workload, adjust within limits, and verify the result. Expanding its scope will require shared scheduling, workload isolation, production reliability, and operator controls.
Starting with local hosting
The workload changes.
The settings follow.
Explore a local host with 24 GiB of GPU memory. Choose a workload to see an example of how context limits, request slots, and memory allocation could adapt.
Allocated memory Available capacity
Baseline
Start with a known configuration.
One resident model, a reserved KV cache, and four request slots. This is an illustrative starting point before the host responds to a change in workload.
Keep a baseline to compare each adjustment against.
Context limit 8,192 tokens per request
- GPU memory allocated
- 16.0 GiBModel weights, KV cache, and runtime buffers
- Request slots allocated
- 4Illustrative concurrent request capacity
- Memory headroom
- 8.0 GiBWithin the same 24 GiB GPU memory budget
An illustration of automatic adjustments. These are example settings, not a live host or measured performance results.
About this example
This synthetic host has 24 GiB of GPU memory and 8 GiB of resident model weights. Allocated memory is the sum of weights, reserved KV cache, and runtime buffers. Each workload uses an assumed cache reservation and context limit; these are not calculated from a specific model architecture. Real requirements depend on the model, precision, runtime, and active tokens. No speed or latency improvement is predicted.
Our approach
Less manual tuning.
Clearer control.
The aim is simple: host a model, give it sensible limits, and let strided adjust the runtime as conditions change. Automation should make local hosting easier to understand and operate.
Let the runtime adjust.
As prompts and request volume change, strided will adjust batching, concurrency, and cache use within your configured limits.
Automatic adjustments. Explicit boundaries.
Check what changed.
See the settings before and after an adjustment, along with the signals that prompted it. Compare the result and return to the previous configuration when needed.
Visible decisions. A way back.
Automatic adjustments. Visible decisions.
When a change cannot be justified, keep the current settings.Signals behind the adjustments
See what changes.
Understand the response.
Longer prompts and more requests change the memory a local model needs. These illustrative traces show the signals that could guide an automatic adjustment.
Time-aligned signalsSampled every 5 seconds
68%
GPU memory used / mean
51%
Context budget used / mean
GPU memory usedContext budget used
Model weightsKV cacheRuntime buffers
View sample data
| Time (s) | GPU memory used (%) | Context budget used (%) |
|---|---|---|
| 0 | 48 | 12 |
| 5 | 50 | 18 |
| 10 | 54 | 25 |
| 15 | 58 | 32 |
| 20 | 62 | 40 |
| 25 | 67 | 48 |
| 30 | 71 | 55 |
| 35 | 76 | 63 |
| 40 | 80 | 70 |
| 45 | 83 | 78 |
| 50 | 85 | 84 |
| 55 | 86 | 88 |
Research
Built around
your local setup.
We're developing local model hosting with automatic runtime tuning. Help us understand where your current setup needs attention: fitting a model into memory, handling longer prompts, or keeping up with concurrent requests.
Share your workloadResearch preview Accepting workloads
Today
Development and workload research with people running models on their own hardware.
Direction
Start with local model hosting, expand toward shared model serving, and ultimately pursue coordinated control of data center systems.
Together
Prefer a conversation? Get in touch Share a capture from your local model, the hardware it runs on, and what you'd like to improve. We'll review it with you.