Tithi MondalBlog

Guide8 min readUpdated

Run AI Locally: Keeping Client Work Private With Ollama

Run AI locally to keep client briefs and unreleased products off outside servers. What Ollama does, the hardware reality, and what never to paste online.

Read inहिन्दीবাংলাमराठीதமிழ்తెలుగుગુજરાતી

Run AI Locally: Keeping Client Work Private With Ollama: cover illustration

Suppose a founder in Siliguri sends a designer the brief for a loan app that has not launched. The 14-page document names the partner bank, the interest rates and the full Know Your Customer (KYC) flow. The designer wants a quick summary and a list of open questions. The fastest way is to paste the document into an online chatbot. At that moment the document leaves the laptop and travels to a server owned by another company.

To run AI locally means to download an AI model to your own computer and use the model there. The text you type is processed on your own machine. This guide explains how that works with Ollama, what hardware to expect, and which things should never go into any online AI tool.

Checked in November 2026. Tools change quickly, so confirm details on the project's own page.

What does it mean to run AI locally?

Running AI locally means the model is a file on your computer, and your computer does the thinking. No internet connection is needed for the answer itself.

Two terms help here. A large language model (LLM) is a program trained on a very large amount of text so that it can write and summarise. A local LLM is the same kind of model, stored and run on your own machine.

Ollama is a program that downloads open models and runs them on your computer. According to the project's documentation, Ollama is available for macOS, Windows and Linux. The README names model families such as Gemma, Qwen and DeepSeek. The repository states that Ollama is released under the MIT licence. Each model has its own separate licence, so read the licence of the model you choose.

How does Ollama work, in simple steps?

Ollama works in four steps: install the program, pick a model, let the model download once, then chat.

  1. Download the installer from the official Ollama website and install the program.
  2. Choose an open model that Ollama offers. Start with a small model.
  3. Start the model. The README shows a short command for this: the word "ollama", the word "run", then the name of the model.
  4. Type your question. The answer is produced on your machine.

Two technical details from the official documentation are useful for a non-developer. First, the README says Ollama has an interface that other programs on your computer can use, at a local address on port 11434. Second, the Ollama FAQ says the program binds to the address 127.0.0.1 by default. In plain words, only your own computer can reach the model unless somebody changes that setting. I would leave that setting alone.

Is a local LLM completely private?

A local LLM keeps your prompts on your machine, but only when the model really runs on your machine. Check this point carefully, because Ollama offers both local and cloud options.

The Ollama FAQ says: "Ollama runs locally. We don't see your prompts or data when you run locally." The same FAQ describes cloud models, which run on Ollama's servers and not on your laptop. The FAQ says Ollama can run in local only mode when the cloud features are disabled, and that you then lose the cloud models and web search. Ollama makes privacy statements about its cloud as well. For confidential client work I would still choose a model that runs on my own machine, because then the question does not arise.

Private AI for business also depends on ordinary computer hygiene:

  • A shared office computer is not private, whatever software runs on the computer.
  • A laptop without a screen lock or disk encryption can leak everything if the laptop is lost.
  • Chat history saved on the laptop is now one more confidential file to protect.
  • An add-on that connects the local model to the internet can send data out again.

Why does keeping client data out of AI tools matter?

Keeping client data out of AI tools matters because a design brief often contains the most sensitive facts of a business before launch. The designer receives those facts in trust.

Think about what arrives in a normal project. A brief with unreleased features and prices. Screenshots of a competitor analysis. Sample user records for a dashboard. A non-disclosure agreement (NDA), which is a written promise not to share the client's information. Fintech work, such as the loan and tax apps I have designed, involves screens that show identity documents and income details. A leak of such material harms the client first and the designer's name second.

When a file goes to an online tool, three questions follow. Where is the file stored? Who can read the file? Is the file used to train future models? The answers differ by tool, by plan and by setting, and the answers change. Read the tool's own data policy. For what your contract requires, ask a qualified professional. My guide to design contract and NDA basics explains what such agreements usually cover.

What hardware does a local LLM need?

A local LLM needs enough memory to hold the whole model, and the experience improves a lot with a good graphics card. A recent laptop can run small models. Large models need a powerful desktop.

In general terms, expect this:

  • Model files are large. One model often takes several gigabytes of disk space, and the first download takes time on a slow connection.
  • Memory decides what you can run. If the model does not fit in memory, the model will not run or will run very slowly.
  • A graphics card speeds things up. According to the project's documentation, Ollama supports Nvidia and AMD graphics cards, and Apple devices through Apple's Metal technology. The FAQ says a model that fits entirely on one graphics card is loaded on that card.
  • Without a graphics card, answers come slowly. A long summary can take minutes on an ordinary office laptop.
  • Small models make more mistakes. A small local model writes weaker text and loses track of long documents more easily than the large online tools.

The documentation pages I read give no single memory figure, because the need depends on the model. Check the page of each model before you download. The software is free, but the hardware, electricity and setup time are real costs.

What should you never paste into an online AI tool?

Never paste anything into an online AI tool that you would not email to a stranger. That one test covers most cases. The specific list is below.

Never pasteDo this instead
Material covered by an NDA, such as unreleased screens, feature lists and pricesDescribe the problem in general words, or use a local model
Personal data of real users: names, phone numbers, Aadhaar or PAN numbers, bank details, KYC photosReplace real records with invented sample data
Passwords, OTPs, login links and access keysNever share these with any tool or person
Customer lists, sales figures and contractsShare only the structure, for example the column names
Private WhatsApp chats and emails from a clientWrite your own short summary without names
Interview recordings and transcripts with real namesRemove names first, and ask the participant's permission

Two simple habits make this easy. First, anonymise before you paste: write "Client A" and "a lending app", not the real names. Second, ask the client in writing before any of their material goes into any AI tool, local or online. I would put that line in the project agreement, so nobody has to guess.

How can a small team use a private model this week, and what are the limits?

A small team can test a private model in one afternoon, on one low-risk task. The model protects the data, but the model does not make the answers correct, so a person checks everything.

  1. Install Ollama on one laptop and download one small model.
  2. Take an old brief from a finished project. Remove the names.
  3. Ask the model for a one-page summary and a list of 10 open questions for the client.
  4. Compare the summary with the brief, line by line. Mark every wrong or invented statement.
  5. Decide which tasks the model handled well enough. Write those tasks down as a team rule.

Good first tasks are summaries, lists of open questions, first drafts of button and error text, and rewording of notes. My guide on how to write a design brief shows what a complete brief contains, which helps you judge the summary.

The honest limits are these. Every language model can state invented facts in a confident voice.

  • Made-up answers. A summary can add a feature that the brief never mentioned. A person must check every summary against the source.
  • Bias. A model repeats the patterns in its training text. Advice about "typical users" may not match users in tier-2 and tier-3 cities of India.
  • Indian languages. Small models are usually weaker in Bengali and Hindi than in English. A native speaker must review such text.
  • Maintenance. Somebody has to update the program and the models, and keep the laptop safe.
  • No responsibility. The model does not know your client and does not answer for the result. The designer does.

A local model is an assistant for drafts. Talking to users, choosing what to build first and standing behind the final screens stay with a person, as I explain in how to use AI with your designer and in what AI design tools can and cannot do. If you have a confidential product to design and want to know how I would handle your files, you are welcome to get in touch.

Sources

Frequently asked questions

Is Ollama free to use?

+

The Ollama repository states that the program is released under the MIT licence, which is an open source licence. Each model you download has its own licence, and any cloud features have their own terms, so check the current details on the official site.

Does a local AI model need the internet?

+

You need the internet to download the program and the model. After that, a model that runs on your own computer produces answers without sending your prompt to an outside server.

Can I run a local LLM on a normal laptop?

+

A recent laptop can run small models, but answers come slowly without a good graphics card. Larger models need more memory, so check the page of each model before you download.

Is it safe to paste a client brief into an online chatbot?

+

Not without the client's written permission. Remove names and confidential details first, read the tool's data policy, or use a model that runs on your own computer.

Working on something like this?

Tell Tithi what you are planning. You get a clear scope, a fixed price and a timeline before any work begins. I reply to every enquiry within 24 hours.

Your details go only to Tithi and are used only to reply to you. No newsletters, no sharing.