Custom Provider

Cherry Studio not only integrates mainstream AI model services, but also gives you powerful customization capabilities. Through Custom AI Provider feature, you can easily connect to any AI model you need.

Why do you need a custom AI provider?

• Flexibility: No longer limited by the preset provider list, freely choose the AI model that best fits your needs.

• Diversity: Try AI models from different platforms and discover their unique advantages.

• Controllability: Directly manage your API keys and access address to ensure security and privacy.

• Customization: Connect to privately deployed models to meet the needs of specific business scenarios.

How do you add a custom AI provider?

With just a few simple steps, you can add your custom AI provider in Cherry Studio:

1. Open Settings: In the left sidebar of the Cherry Studio interface, click “Settings” (gear icon).

2. Go to Model Services: On the settings page, select the “Model Services” tab.

3. Add provider: On the “Model Services” page, you will see a list of existing providers. Click the “+ Add” button below the list to open the “Add Provider” dialog.

4. Fill in the information: In the dialog box, you need to fill in the following information:

   • Provider Name: Give your custom provider an easily recognizable name (for example: MyCustomOpenAI).

   • Provider Type: Select your provider type from the drop-down list. Currently supported:

     • OpenAI

     • Gemini

     • Anthropic

     • Azure OpenAI

5. Save configuration: After filling it out, click the “Add” button to save your configuration.

Configure the custom AI provider

After adding it, you need to find the provider you just added in the list and configure it in detail:

1. Enable status There is an enable switch on the far right of the custom provider list; turning it on means enabling this custom service.

2. API key:

   • Enter the API key provided by your AI provider (API Key).

   • Click the “Check” button on the right to verify the validity of the key.

3. API address:

   • Enter the API access address of the AI service (Base URL).

   • Be sure to refer to the official documentation provided by your AI provider to obtain the correct API address.

4. Model management:

   • Click the “+ Add” button to manually add the model ID you want to use under this provider. For example gpt-3.5-turbo,gemini-pro etc.

   

   • If you are not sure of the exact model name, please refer to the official documentation provided by your AI provider.

   • Click the "Manage" button to edit or delete models that have already been added.

Get started

After completing the above configuration, you can select your custom AI provider and model in Cherry Studio’s chat interface and start chatting with the AI!

Using vLLM as a custom AI provider

vLLM is a fast and easy-to-use LLM inference library similar to Ollama. Here are the steps to integrate vLLM into Cherry Studio:

1. Install vLLM: Follow the official vLLM documentation (https://docs.vllm.ai/en/latest/getting_started/quickstart.html) to install vLLM.

   pip install vllm # if you use pip
   uv pip install vllm # if you use uv

2. Start the vLLM service: Use the OpenAI-compatible interface provided by vLLM to start the service. There are mainly two ways, as follows:

   • Usevllm.entrypoints.openai.api_serverStart

   python -m vllm.entrypoints.openai.api_server --model gpt2

   • UseuvicornStart

   vllm --model gpt2 --served-model-name gpt2

Make sure the service starts successfully and is listening on the default port 8000 . Of course, you can also specify the port number of the vLLM service through the parameter--port.

1. Add the vLLM provider in Cherry Studio:

   • Follow the steps described earlier to add a new custom AI provider in Cherry Studio.

   • Provider Name: vLLM

   • Provider Type: Select OpenAI.

2. Configure the vLLM provider:

   • API key: Since vLLM does not require an API key, you can leave this field blank or fill in any content.

   • API address: Enter the API address of the vLLM service. By default, the address is: http://localhost:8000/(if you use a different port, modify it accordingly).

   • Model management: Add the model name loaded in vLLM. In the example run abovepython -m vllm.entrypoints.openai.api_server --model gpt2you should enter heregpt2

3. Start chatting: Now, you can select the vLLM provider and gpt2 model in Cherry Studio and start chatting with the LLM powered by vLLM!

Tips and tricks

• Read the documentation carefully: Before adding a custom provider, be sure to carefully read the official documentation of the AI provider you are using to understand key information such as API keys, access addresses, and model names.

• Check the API key: Use the “Check” button to quickly verify the validity of the API key and avoid being unable to use it due to an incorrect key.

• Pay attention to the API address: Different AI providers and models may have different API addresses, so be sure to enter the correct one.

• Add models as needed: Please only add the models you will actually use, and avoid adding too many unnecessary models.