Local LLM Integration with ColdFusion: Running AI Models via Ollama

If your data can’t leave your network – PHI, financial records, anything covered by a strict data processing agreement – a local model removes the “can we send this externally” question entirely.

Setting up Ollama

Ollama runs open-source models (Llama, Mistral, and others) on your own server. After installing it and pulling a model:

ollama pull llama3.1:8b
ollama serve

Ollama exposes a local API (by default at http://localhost:11434) that your ColdFusion application connects to.

Option A: Native ChatModel() (ColdFusion 2025 Update 8)

ChatModel() also supports local models via Ollama — the configuration is similar, with an additional baseUrl parameter pointing to your local endpoint:

chatModelConfig = {
    provider    : "ollama",  // confirm exact provider string in your CF Administrator AI panel
    modelName   : "llama3.1:8b",
    baseUrl     : "http://localhost:11434",
    temperature : 0.3,
    maxTokens   : 500,
    timeout     : 60
};

chatModel = ChatModel( chatModelConfig );
response  = chatModel.chat( "Classify this support ticket as billing, technical, or general: " & ticketBody );

writeOutput( encodeForHtml( response.message ) );

Local models are typically slower per request than a cloud frontier model unless you have a capable GPU behind them, so the timeout value above is intentionally more generous than the cloud examples elsewhere on this blog.

Option B: Self-managed deployment (any other supported version, or Lucee)

On older ColdFusion versions, Ollama still runs the same way – the difference is you call its REST API directly with cfhttp instead of the native ChatModel() wrapper, following the same JSON request/response pattern used for the other providers on this blog.

Setting expectations on capability

This is the honest part: an 8B-parameter local model is not going to match GPT-5 or Claude Sonnet on complex reasoning tasks. Where local models genuinely hold up well is narrower, well-defined work – classification against a fixed set of categories, structured extraction, straightforward summarisation. Test against your actual data before committing a production workflow to a local model, rather than assuming capability parity with a frontier model.

Frequently Asked Questions

What hardware do I need to run this at acceptable speed? 

Depends on model size – a 7–8B parameter model runs reasonably on a modern GPU with 16GB+ VRAM; larger models need proportionally more. CPU-only inference works but is noticeably slower. 

Can I run multiple models and route between them? 

Yes – Ollama supports multiple pulled models, and your ColdFusion integration can select which one to call per request based on task type or data sensitivity. 

Is this actually more private than a cloud API? 

Yes, meaningfully – with a local model, the data in your prompt never leaves your infrastructure, which is the entire point for compliance-constrained use cases. 

For help scoping a local/private AI deployment for your ColdFusion application, see our AI Integration Services with ColdFusion page.