LocalAI Embeddings

LocalAI Setup

LocalAI is a drop-in replacement REST API that’s compatible with OpenAI API specifications for local inferencing. It allows you to run LLMs (and not only) locally or on-prem with consumer grade hardware, supporting multiple model families that are compatible with the ggml format.

To use LocalAI Embeddings within Flowise, follow the steps below:

1. git clone https://github.com/go-skynet/LocalAI

cd LocalAI

3. LocalAI provides an [API endpoint](https://localai.io/api-endpoints/index.html#applying-a-model---modelsapply) to download/install the model. In this example, we are going to use BERT Embeddings model:

<figure><img src="/images/flowise/image (27) (1).png" alt="" /><figcaption></figcaption></figure>

4. In the `/models` folder, you should be able to see the downloaded model in there:

<figure><img src="/images/flowise/image (23) (1) (2).png" alt="" /><figcaption></figcaption></figure>

5. You can now test the embeddings:

```bash
curl http://localhost:8080/v1/embeddings -H "Content-Type: application/json" -d '{
    "input": "Test",
    "model": "text-embedding-ada-002"
  }'

6. Response should looks like:

Flowise Setup

Drag and drop a new LocalAIEmbeddings component to canvas:

Fill in the fields:

• Base Path: The base url from LocalAI such as http://localhost:8080/v1
• Model Name: The model you want to use. Note that it must be inside /models folder of LocalAI directory. For instance: text-embedding-ada-002

That’s it! For more information, refer to LocalAI docs.