Together AI integration | LangChain 日本語

Warning

You are currently on a page documenting the use of Together AI models as text completion models. Many popular Together AI models are chat completion models.

You may be looking for this page instead.

Together AI offers an API to query 50+ leading open-source models in a couple lines of code.

This example goes over how to use LangChain to interact with Together AI models.

Installation

pip install -U langchain-together

Environment

To use Together AI, you'll need an API key which you can find here: api.together.ai/settings/api-keys. This can be passed in as an init param together_api_key or set as environment variable TOGETHER_API_KEY.

Example

# Querying chat models with Together AI

from langchain_together import ChatTogether

# choose from our 50+ models here: https://docs.together.ai/docs/inference-models
chat = ChatTogether(
    # together_api_key="YOUR_API_KEY",
    model="meta-llama/Llama-3-70b-chat-hf",
)

# stream the response back from the model
stream = chat.stream_events("Tell me fun things to do in NYC", version="v3")
for token in stream.text:
    print(token, end="", flush=True)

# if you don't want to do streaming, you can use the invoke method
# chat.invoke("Tell me fun things to do in NYC")

# Querying code and language models with Together AI

from langchain_together import Together

llm = Together(
    model="codellama/CodeLlama-70b-Python-hf",
    # together_api_key="..."
)

print(llm.invoke("def bubble_sort(): "))