Buying Compute

Under construction

Topping up your account

  1. Coming soon

Access via chat interface:

  1. Click Buy > AI Chat when logged in. Inference is billed for the currently loaded model.

Access via openai-compatible endpoint:

  1. Click Buy > API Key when logged in
  2. Copy the API key and use it wherever you would use an OpenAI API key
  3. Change the base url to https://queenbee.gputopia.ai/v1

Some codebases append the /v1 in code, so you may need to try just https://queenbee.gputopia.ai

Supported interfaces

  • Openai compatible Fine Tuning, Chat Inference and Embeddings are currently supported

Model names

  • Use hugging face model names, or your own fine-tuned model name (for now)
  • Feel free to use a publicly available URL to any GGUF file for an inference model name instead (our workers will download/run it)
  • Use fastembed:BAAI/bge-base-en-v1.5 (or any other fastembed model) for very fast /v1/embeddings endpoint use
  • Use repo/model:filter format for any hugging face model.
    • A worker will load your model.
    • First time usage will be slow as the model propagates to different workers
    • The more you use it, the faster it will get (will gain priority over time)
  • For example: TheBloke/zephyr-7B-beta-GGUF:Q5_K_M works fine, and is comparable, in many ways, to gpt3

Python example code:

import openai

# Set up the OpenAI client

openai.api_key = "YOUR_ACCESS_TOKEN"
openai.api_base = "https://queenbee.gputopia.ai/v1"

# Use the model with the chat completion endpoint

response = openai.ChatCompletion.create(
model="TheBloke/vicuna-7B-v1.5-GGUF:Q4_K_M",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Translate the following English text to French: 'Hello World'"},
]
)

print(response.choices[0].message['content'].strip())

Node example code:

const OpenAIApi = require('openai');

const openai = new OpenAIApi({
  apiKey: 'YOUR_GPUTOPIA_API_KEY',
  baseURL: 'https://queenbee.gputopia.ai/v1'
});

async function getCompletion() {
  const response = await openai.chat.completions.create({
    model: "TheBloke/vicuna-7B-v1.5-GGUF:Q4_K_M",
    messages: [
      {role: "system", content: "You are a helpful assistant."},
      {role: "user", content: "Translate the following English text to French: 'Hello World'"}
    ]
  });

  console.log(response.choices[0].message.content.trim());
}

getCompletion();