Call
Talk to an expert

Secure AI

Local AI & Private LLMs

A hardened open-weight model and private search over your own documents, piloted with a sample of your files before you commit.

Ask about Local AI & Private LLMs
Your private AI deploymentSample view
  • Model hostOutbound internet access blockedManaged
  • HR assistantFinance document request declinedContained
  • Chat interfaceSign-in through single sign-on and MFAManaged
  • Model filesChecksum verified against sourceManaged
  • Prompt logsRetention period to approveDecision needed
  • Document indexFolder permissions carried into searchManaged
Illustration · not live data

In short

Local AI means running open-weight language models on your own hardware or in an Australian private cloud, so prompts and documents stay on systems you control. Catharsis, a Sydney cyber security company certified to ISO/IEC 27001:2022, designs, deploys and hardens private AI, including private search and question answering over your company documents.

01Who it's for

Is local AI right for you?

It suits organisations that can't send client or sensitive information to public AI services, or must keep data in Australia, but still want AI over their own documents.

Built for

  • Legal, health, accounting and finance practices
  • Government, council and education teams
  • Businesses with 5 or more staff that want AI over internal documents without a per-user cloud subscription
  • Organisations whose own security policy restricts where data may be processed

You probably need it if…

  • Client confidentiality or professional rules stop you using public AI for client work
  • Government, health or education requirements say data must stay in Australia
  • Customer contracts restrict where their data may be processed
  • You want staff to search internal documents with AI without sending them to a third party

02What's included

What local AI includes

Design & deploy

Use-case and sizing workshop

We agree what the AI needs to do, how many people will use it and which documents it should draw on. That sets the model size, the hardware and whether on-premises or private cloud suits you.

Model selection and deployment

We choose open-weight models suited to your task and licence needs, then deploy them on your hardware or in an Australian private cloud, with a simple chat interface and an internal application programming interface (API) for other tools.

Private RAG over your documents

Retrieval-augmented generation (RAG) lets staff ask questions and get answers drawn from your policies, contracts, manuals and knowledge base, with links to the source documents. Retrieval is designed to respect who is allowed to see each document.

Harden & control

Hardening and network isolation

The model host is isolated on your network, outbound internet access is blocked or tightly limited, and the interface sits behind single sign-on and multi-factor authentication (MFA). We disable telemetry and remove services the deployment doesn't need.

Logging and access control

Prompts, responses and document retrievals are logged to your monitoring, with retention you choose. Access is granted by role, so an AI assistant for HR can't answer questions from finance documents.

Hardware

Hardware supply and configuration

If you need a graphics processing unit (GPU) workstation or server, we source it, configure it securely and install it through our IT hardware service, so one team is responsible from the box to the working assistant.

Close-up of a server with rows of drive bays and green status lights, representing on-premises hardware for local AI

Secure AI

Keeping your document index as private as the documents

Private search builds an index from your files, and that index can reveal what the files say. We design search to respect your existing folder permissions, so answers are drawn from documents staff are already allowed to open.

03In detail

Local AI & private LLMs, in depth

Why does private RAG need its own access controls?

Retrieval-augmented generation (RAG) splits your documents into passages and indexes them, so the model can answer from your own content. For example, an HR assistant could answer “How much parental leave do we offer?” from your current policy, with a link to the clause, while declining to use documents the person asking isn’t allowed to open.

That raises the main security question for any RAG system: who can see what. A RAG system that ignores document permissions can hand a junior staff member a summary of a board paper. The OWASP Top 10 for LLM Applications lists vector and embedding weaknesses as a distinct risk for exactly this reason, so we build retrieval to respect the same access rules as the original documents.

How do we check the model itself is safe to run?

Open-weight models are downloaded files, so they carry supply chain risk (LLM03 in the OWASP Top 10 for LLM Applications). We take model files only from their official source, check them against published checksums and use safe formats such as safetensors. The rest of the hardening treats the model host like any other sensitive server.

The framework mapping below shows how these controls line up with ISO/IEC 27001 and ISO/IEC 42001, which matters if you report to a board or a government buyer. Our AI security and governance service explains those standards and can add the policy and risk register around the deployment.

Should you buy hardware or use private cloud?

If buying hardware suits you, our IT hardware procurement service sources the workstation or server, configures it securely before delivery and installs it. If you would rather not buy hardware, a dedicated environment in an Australian private cloud is the alternative. We recommend the provider and region after discussing your needs.

If you are still deciding whether local AI is the right path at all, our AI consulting service compares it with cloud options for your use case.

04The process

How local AI works

  1. Scope and size

    We agree use cases, users and documents, then recommend a model size, hardware and hosting option with a written quote.

  2. Pilot

    We deploy a pilot with a sample of your documents so a small group can test answer quality on real questions.

  3. Harden and test

    We lock down the host, connect logging and test for prompt injection and access-control bypass before wider use.

  4. Roll out

    The assistant goes live for the agreed users, with a short usage policy and training.

  5. Maintain

    If you choose ongoing support, we patch the host and software, review logs and update models and document indexes as your needs change.

05Frameworks

How this maps to the frameworks

Useful for your auditor, board or procurement team. The mapping shows where this service supports each framework. It doesn't replace a formal assessment.

Local AI & Private LLMs: framework mapping
ControlASD EssentialsASD maturity model (current)ISO/IEC 27001NIST CSFISO/IEC 42001
Network isolation of the model hostBoundary servicesNot applicableA.8.20, A.8.22Protect (PR.IR-01)Not applicable
Access control for AI and document retrievalIdentity servicesMulti-factor authenticationA.5.15, A.8.3, A.8.5Protect (PR.AA-05)Not applicable
Model host and software patchingNot applicablePatch applicationsPatch operating systemsA.8.8Protect (PR.PS-02)Not applicable
Prompt and retrieval loggingVisibility and responseNot applicableA.8.15Detect (DE.CM-09)A.6.2.8
Model provenance and data sourcesNot applicableNot applicableA.5.21Govern (GV.SC-07)A.7.5

06Pricing

Pricing and how to start

Pricing

Scoped after a strategy call

What changes the price

Price depends on the number of users and assistants, the volume and type of documents, the model size, whether hardware is bought or private cloud capacity is rented, and whether you want us to maintain the deployment afterwards. Hardware is quoted separately.

  • Use-case and sizing workshop
  • Deployment of an open-weight model with a chat interface
  • Private RAG over an agreed set of documents
  • Hardening, single sign-on and logging set-up
Book a strategy call

07FAQ

Local AI & private LLMs, answered.

Can't see your question? Ask a specialist and get a straight answer.

Ask a specialist
What is a private LLM?

A private LLM is a large language model that runs on infrastructure you control, such as your own server or a dedicated private cloud environment, instead of a public AI service. Prompts, documents and answers stay within your environment. Private LLMs usually use open-weight models, whose model files are published for organisations to download and run themselves under the model's licence.

Is a local AI model as capable as ChatGPT or Copilot?

Usually not for every task, but often good enough for the tasks that matter to you. The largest public models remain more capable for broad, open-ended work. For focused jobs, such as answering questions from your own documents, summarising or drafting, a well-chosen open-weight model can do the job. The pilot tests this on your real questions before you commit.

What is private RAG?

Private RAG, or retrieval-augmented generation, connects a language model to your own documents. When a staff member asks a question, the system finds the most relevant passages, gives them to the model and returns an answer with links to the sources. Everything runs on your infrastructure, and retrieval is designed to follow document permissions, so people get answers only from documents they are allowed to see.

Does our data stay in Australia with local AI?

Yes, when the model runs on your own hardware in Australia or in a private cloud environment hosted in Australia. We design the deployment so prompts, documents and logs are stored and processed only there, and we block outbound connections the system doesn't need. You can verify this, because the infrastructure is yours or dedicated to you.

What hardware do we need to run local AI?

It depends on the model size and the number of people using it at once. Small pilots can run on a single workstation with a capable graphics processing unit (GPU). Larger models and more users need a dedicated server with more GPU memory. We size the hardware after the use-case workshop, then source and configure it for you.

How do you secure a local AI deployment?

We treat the model host like any other sensitive server: network isolation with restricted outbound access, single sign-on with MFA, access by role, logging, patching and verified model files. Before staff use it, we try to break it, including attempts to get around document permissions.

Talk to our team

Talk to us about local AI & private LLMs.

No obligation. You talk to the people who do the work.