Recommendations: Connecting LLMs to External Tools (MCP)

1. Industry Standard Approaches

a. Function Calling / Tool Use APIs

• OpenAI Function Calling: Lets you define functions (tools) that the LLM can invoke by returning a structured JSON object. The orchestrator then executes the function and returns results to the LLM or user.

• Anthropic Tool Use: Similar to OpenAI, lets you define tools and the LLM will output tool calls in a structured format.

• Google Gemini, Microsoft Copilot, etc.: All major LLM providers are adding this “function/tool calling” capability.

Key Point: The industry standard is to use a function-calling interface where the LLM emits a structured tool call, which your code executes, and then the result is returned to the user (optionally with LLM summarization).

---

2. Existing Libraries & Frameworks

a. Python Ecosystem

• LangChain: The most popular open-source framework for agentic LLMs with tool/function calling, routing, and orchestration.

• LlamaIndex: Similar to LangChain, with focus on data and tool augmentation.

• OpenAI SDK: Supports function calling out of the box.

b. Go Ecosystem

• Less mature than Python, but options exist:

  • go-openai: Supports OpenAI function calling.

  • ollama-go: For Ollama, but native function calling is not yet as advanced as OpenAI’s.

  • Custom orchestration: Many Go projects implement their own “function calling” bridge, parsing the LLM’s output for tool calls and executing them.

---

3. What If My LLM Doesn’t Support Function Calling?

• Prompt Engineering: Prompt the LLM to output tool calls in a specific format (e.g., JSON or a command string), then parse and execute them.

• Custom Middleware: Write a layer that detects when a tool call is needed, either by parsing the LLM’s output or by using rules/regex.

---

4. My Recommendation for Your Project

If you want industry best practices:

• Use an LLM that supports function/tool calling (OpenAI GPT-4/3.5, Anthropic, etc.).

• Define your MCP tools as “functions” with schemas.

• Use a library/framework (LangChain, go-openai, etc.) to handle orchestration.

If you want to stick with Go and Ollama:

• Implement a simple function-calling bridge:

  1. Prompt the LLM to output tool calls in a structured format (e.g., {"tool": "filesystem", "action": "list", "path": "/Users/tuannvm/Downloads"}).

  2. Parse the LLM output for tool calls.

  3. Execute the tool call via MCP.

  4. Return the result to the user (optionally letting the LLM summarize it).

If you want Python and maximum flexibility:

• Use LangChain or LlamaIndex with OpenAI/Anthropic for function calling and tool orchestration.

---

5. Summary Table

Approach	LLM Required	Library/Framework	Language	Pros	Cons
OpenAI Function Call	GPT-3.5/4	go-openai, LangChain	Go/Python	Standard, robust	Needs OpenAI API
Anthropic Tool Use	Claude	LangChain, SDK	Python	Standard, robust	Needs Anthropic API
Custom Bridge	Any	None / custom	Go	Flexible, works now	Manual parsing
LangChain	Any (best w/FC)	LangChain	Python	Agentic, batteries	Python only

---

6. Links

• OpenAI Function Calling Docs
• LangChain Tool Use
• go-openai Function Call Example

---

7. What Should You Do?

• If you want to stay in Go and with Ollama, build a simple tool-calling bridge as described above.

• If you want maximum flexibility and industry alignment, consider using OpenAI/Anthropic with function calling and a framework like LangChain.

Let me know which approach you’d like to pursue, and I can help you implement it step by step!