llama-3.1-70b
Workhorse for chat and RAG. Streaming, function calling, JSON mode.
Context128K
Input75 ₽/M
Output240 ₽/M
Tok/sec240
OpenAI-compatible API on our GPUs in-country. Llama 3.1, Qwen 2.5, Mixtral — streaming, function calling, JSON mode. Fine-tuning on B300. FZ-152 compliant. No vendor lock-in and your data doesn't leave the country.
Llama · Qwen · Mixtral · embeddings · fine-tuning · FZ-152
Frontier models from OpenAI and Anthropic are great, but they live abroad. Open-source models aren't one click away. Your own VM with vLLM is brittle. We built what sits in the middle.
Same endpoint, same SDK. curl with Authorization-bearer, Python openai with base_url — either way, first token in ~80 ms with GPUs in-country.
# drop-in openai api replacement
$ curl https://api.h3llo.cloud/v1/chat/completions \
-H "Authorization: Bearer h3llo_•••" \
-H "Content-Type: application/json" \
-d '{
"model": "llama-3.1-70b",
"stream": true,
"messages": [
{"role": "user", "content": "Explain Raft in 3 sentences"}
]
}'
# ~80 ms later — first token:
data: {"choices":[{"delta":{"content":"Raft"}}]}
data: {"choices":[{"delta":{"content":" is a"}}]}
data: {"choices":[{"delta":{"content":" consensus algorithm..."}}]}# the same openai SDK, only base_url changed
from openai import OpenAI
client = OpenAI(
base_url="https://api.h3llo.cloud/v1",
api_key="h3llo_•••",
)
resp = client.chat.completions.create(
model="llama-3.1-70b",
messages=[{"role": "user", "content": "..."}],
stream=True,
)
for chunk in resp:
print(chunk.choices[0].delta.content, end="")Prices in rubles, per-second billing. Context length is the actual one — no hidden caps. TPS measured at batch-size 1, int8 quantization, B300 MIG 141 GB.
Workhorse for chat and RAG. Streaming, function calling, JSON mode.
Frontier-class for the hardest tasks. Reasoning, long chains, agents.
Cheapest model in the catalog. Ideal for batch inference and simple tasks.
Strong on code and multilingual. An alternative to Llama 70B with better Chinese/Russian.
Mid-class — balance of price and quality. Good for agents and tool use.
Mixture-of-Experts: 141B params, 39B active. Good on reasoning and multi-turn.
Multilingual embeddings for RAG. 4 096-dim vector, sentence-aware.
Cheap multilingual embeddings model — 100+ languages, 1 024-dim vector.
Embedding models + chat inference with function calling. p99 ≤ 800 ms on a corpus of 4M documents.
Function calling, JSON mode, structured outputs. Llama 70B and Qwen 72B hold up in complex multi-turn conversations.
Upload jsonl, kick off a job on B300. You get an endpoint with your version of Llama. From 4 hours.
Async jobs for millions of requests. 4× cheaper inference when latency isn't critical for the task.
Per-token billing for prototypes and variable load. Reserved GPU for predictable invoices. Dedicated — for regulated markets.
Real practices, benchmarks, case studies. No fluff and no marketing — grab the PDF, apply it Monday.
Llama 3.1 70B, Qwen 2.5: TPS, p99 latency, cost per 1M tokens across batches. ~15 min.
→Guide · 28 ppHow we hold p95 ≤ 800 ms on a corpus of 4M documents. With formulas and code. ~30 min.
→Case · HyperionWhat we measured, what we optimized, which batches we hold. Real before/after numbers. ~25 min.
→Checklist · PDFDataset quality, validation, eval metrics, rollback. What you must check. ~20 min.
→Guide · 22 ppThreat model for a production LLM service. With examples and priorities. ~25 min.
→Templates · GitHubProduction-ready: gateway, OpenAI-compat proxy, Qdrant, Grafana. For dev/staging/prod.
→No SDK to rewrite. No new API to learn. Just change base_url — and your code runs on our models in-country.
`h3llo auth login` · `h3llo ai key create` — in 30 seconds.
`https://api.openai.com/v1` → `https://api.h3llo.cloud/v1`. Done.
`model: "llama-3.1-70b"` instead of `gpt-4`. Streaming works.
Your code doesn't change — only the endpoint. Function calling, JSON mode, embeddings — all there.
No migrations, no rewrites, no vendor lock-in. Get a key, change base_url — it works.