ChatGPT, Microsoft 365 Copilot and private AI on your own hardware handle your data in very different ways. This guide compares them on what each vendor actually commits to, adds measured numbers from our own local AI lab, and ends with an eight-question flow that tells you which fits which job.
TL;DR
- Most organisations shouldn’t pick one. Use Microsoft 365 Copilot for everyday work inside Microsoft 365 once your permissions are clean, a business ChatGPT plan where you need the strongest general model, and private AI for data that must never leave your control.
- OpenAI says it doesn’t train on business ChatGPT plans or the API by default, and Microsoft says Copilot prompts aren’t used to train foundation models. Free and personal ChatGPT accounts are used for training unless the user turns it off, which is where most accidental leaks start.
- Australian residency is moving but not settled: check, in writing, where your plan stores and processes prompts before you put personal information through it.
- Local AI is real but bounded. In our lab a 30B-class model on one NVIDIA DGX Spark generated about 60 to 75 tokens a second, and a second user waited a median of 18 seconds when requests collided.
- The eight-question flow below gives you a recommendation per type of data, not one answer for the whole organisation.
In this post
- What are you actually choosing between?
- How do the three compare?
- What does each vendor say about your data?
- Does the data stay in Australia?
- What can local AI actually do? Numbers from our lab
- Which one fits you? An eight-question decision flow
- The hybrid pattern most organisations end up with
- Before you sign anything: a short checklist
- Key takeaways
- Sources
What are you actually choosing between?
The three names cover six or seven different products, and the data terms differ more between the plans than between the brands.
ChatGPT. OpenAI sells personal plans (Free, Plus, Pro), business workspaces (Business, Enterprise, Edu) and an API for developers. The personal plans are what staff use when nobody has given them anything else.
Microsoft 365 Copilot. Copilot Chat is included with eligible Microsoft 365 licences and is mainly grounded in the web plus whatever the user uploads or pastes. The paid Copilot add-on licence (Microsoft’s newer documentation calls it Microsoft Copilot) reasons across mail, chats, meetings, SharePoint and OneDrive through Microsoft Graph, limited to what each user can already open.
Private AI. Open-weight models running on hardware you control, in your office, a rack or a private hosting arrangement. Nothing is sent to a model vendor. You, or someone you pay, run it.
How do the three compare?
| ChatGPT (business plans) | Microsoft 365 Copilot | Private AI (your hardware) | |
|---|---|---|---|
| Trains on your data? | Not by default on Business, Enterprise, Edu and the API, per OpenAI. Personal plans do unless switched off | No: prompts, responses and Graph data aren’t used to train foundation models | No vendor involved |
| Where data is processed | OpenAI’s cloud. Australian storage at rest for eligible Enterprise, Edu and API customers; OpenAI’s API documentation lists no in-region processing for Australia | Microsoft’s cloud. Outside the EU, prompts may be processed in the US, EU or other regions; Microsoft expects in-country processing in Australia by the end of 2026 | Where the box is |
| Cost model | Per seat per month; API billed per token | Copilot Chat included with eligible licences; full Copilot is a per-user add-on licence | Upfront hardware, power and the operator’s time; no per-token fee |
| Skills you need | Admin settings, policy, staff training | Microsoft 365 permissions, sharing links, sensitivity labels | Linux, GPU drivers, containers, model serving, monitoring, patching, capacity planning |
| Control | Vendor’s admin console and contract | Your tenant’s permissions, labels, retention and audit | Everything, including the failures |
| Capability | Frontier models, web research, image generation, long documents | Frontier models plus your Microsoft 365 content | Smaller open models: good at summaries, extraction, routing and tool use, not frontier reasoning |
| Fits | General writing, research and analysis on non-sensitive data | Organisations already working in Microsoft 365 | Confidential, personal or privileged data; steady, automated workloads |
The details behind each row are below, with links to the vendor’s own pages. Vendor terms change, so check them on the day you decide.
What does each vendor say about your data?
OpenAI. Its business data page says that by default it doesn’t use inputs or outputs from ChatGPT Enterprise, Business, Edu or the API platform to train or improve its models. Its API documentation says abuse-monitoring logs are kept for up to 30 days unless the law requires longer, and that zero data retention is available on some endpoints with OpenAI’s approval. For personal Free, Plus and Pro accounts, OpenAI’s help centre says training is on by default, controlled by a setting called "Improve the model for everyone" under Settings > Data controls. Turning it off applies to new conversations, and the help centre also says that if you give thumbs-up or thumbs-down feedback, the whole conversation may be used even if you’ve opted out. Temporary chats aren’t used for training and are deleted after 30 days. Plan details change, so confirm the training and retention terms in your own contract or order form.
Microsoft. Microsoft Learn states that "prompts, responses, and data accessed through Microsoft Graph aren’t used to train foundation LLMs". Copilot and Copilot Chat used with a work account are covered by Microsoft’s Data Protection Addendum and Product Terms, with Microsoft acting as a processor. Two points matter in practice. First, Copilot "only surfaces organizational data to which individual users have at least view permissions", so a SharePoint site shared with everyone becomes searchable by everyone through Copilot. Second, Microsoft now offers third-party models inside Copilot, which admins can choose to allow, and some of them currently sit outside Microsoft’s EU Data Boundary and in-country processing commitments. If residency matters to you, check which models your tenant allows.
Private AI. No vendor sees the prompt, but nothing protects it either except what you build: access control, logging, encryption, backups and patching. In our lab that meant one gateway as the only way in, a key per application, and a check before deployment that refused any configuration able to send data to a cloud model.
The Australian regulator. The OAIC’s guidance on commercially available AI products recommends that organisations do not enter personal information, particularly sensitive information, into publicly available generative AI tools. It also points to APP 8 when personal information could be disclosed outside Australia. This is general information, not legal advice; if you handle health, legal or government data, get advice on your specific obligations.
Does the data stay in Australia?
It depends on the plan, the feature and the month you check.
- OpenAI‘s help centre lists Australia as a data residency region for eligible ChatGPT Enterprise and Edu workspaces. For the API, OpenAI’s developer documentation lists Australia for regional storage on approved projects but no regional processing, and says that in such regions OpenAI "may also process and temporarily store Customer Content outside of the Region to deliver the services". In other words, stored content can stay in Australia while the model runs elsewhere.
- Microsoft says Copilot is covered by the data residency commitments in its Product Terms, and its Advanced Data Residency and Multi-Geo offerings include Copilot. Its privacy page still says customers outside the EU "may have their queries processed in the US, EU, or other regions". Microsoft announced in-country processing of Copilot interactions for Australia, then updated the timeline in April 2026 to "by the end of 2026". Check the current status for your tenant before relying on it.
- Private AI stays wherever you put the hardware, which is the only option where the answer is guaranteed by physics rather than a contract.
Check: ask your vendor or partner to confirm, in writing, where prompts and responses are stored and where they are processed for your exact plan, and keep that answer with your privacy impact assessment.
What can local AI actually do? Numbers from our lab
We ran a private AI server in our lab for seven weeks on an NVIDIA DGX Spark, a desktop system built on the GB10 Grace Blackwell Superchip with 128 GB of unified memory shared by the CPU and GPU, roughly 120 GB of it usable. It served test workflows, a chat agent and a document pipeline on synthetic and sample data. The full write-up is in our lab series; these are the numbers that matter for a buying decision.
| Measure | What we measured |
|---|---|
| Generation speed, 30B mixture-of-experts coding model at 4-bit, on the GPU | About 60 to 75 tokens a second for one request |
| Same model before the GPU worked, on the CPU | About 15 tokens a second |
| Two models kept loaded, one request slot each, 32,768-token context | 37.8 GB of memory |
| A short direct request | Answered in 2.5 seconds |
| Chat agent replies, after trimming its fixed prompt | 3 to 7 seconds |
| Loading a model after a reboot | 10.5 seconds for one model, about 70 seconds for another |
| Waiting when two callers hit the same model (7 days, 1,342 requests, 52 overlaps) | Median 18 seconds, 90th percentile 58 seconds |
| Largest prompt we admit | 26,000 tokens, leaving room for the answer in a 32,768-token window |
Three lessons from those numbers. Speed for one user is fine: 60 tokens a second is faster than anyone reads. Concurrency is the constraint: with one slot per model, the second person waits, and adding slots costs memory (four slots at a 64k context took one model from about 20 GB to about 33 GB and helped freeze the server). And quality is good rather than frontier: the 30B-class models handled routing, extraction, summaries and tool calls well, but we wouldn’t give them work that needs the strongest reasoning available. If you want to size a box yourself, our memory sizing guide has the formulas and a calculator.
Want AI that keeps your data in-house? We design, build and secure private AI on hardware you control, with the guardrails our lab series describes.
Which one fits you? An eight-question decision flow
Answer these once for each kind of data or task, not once for the whole organisation. A practice might land on Copilot for internal admin and private AI for client files.
- Will the AI see personal information, health information, client-confidential or legally privileged material? No: go to question 5. Yes: go to question 2.
- Does a contract, client, regulator or your own policy say this data must not leave your systems? Yes: private AI for this data. If it only has to stay in Australia, go to question 3. No restriction: go to question 4.
- Will your cloud vendor commit in writing to storing and processing it in Australia on your plan? No: private AI for this data, or keep it out of AI tools. Yes: go to question 4.
- Have you done the APP 8 and privacy due diligence (privacy impact assessment, privacy policy, collection notice, vendor terms)? No: do it first, and keep personal information out of AI tools until it’s done. Yes: go to question 5.
- Does most of the work happen in Microsoft 365: email, Teams, SharePoint, OneDrive? Yes: go to question 6. No: go to question 7.
- Are your SharePoint and OneDrive permissions and sharing links in order? Yes: Microsoft 365 Copilot. No: start with Copilot Chat, fix permissions, then license Copilot for a pilot group.
- Does the work need frontier capability: hard multi-step reasoning, broad world knowledge, web research or image generation? Yes: a ChatGPT business plan (or Copilot, if you’re on Microsoft 365). No: go to question 8.
- Is the workload steady or automated (document pipelines, classification, bulk summaries), and do you have someone to run a server? Yes to both: private AI is a strong fit and gets cheaper per task the more you use it. Otherwise: a ChatGPT business plan, and revisit private AI when volume grows.
Whatever comes out, one rule applies to everyone: block or discourage personal ChatGPT accounts for work, because that is where training is on by default and no admin can see what was shared.
The hybrid pattern most organisations end up with
The pattern that works is to route by data classification, not by tool preference.
| Data class | Example | Route |
|---|---|---|
| Public | Website copy, published policies, general research | Any approved business AI plan |
| Internal | Meeting notes, internal procedures, project email | Copilot, under existing Microsoft 365 permissions |
| Confidential or personal | Client files, HR records, health information, contracts | Private AI on your own hardware, or no AI |
| Restricted | Legal privilege, security credentials, regulator-restricted data | No AI unless specifically assessed and approved |
Three things make it hold:
- One way in. Applications call AI through a single gateway with a key each, and the gateway decides which model a request may reach. In our lab, cloud models were reachable only by named keys, and a pre-deployment check refused any configuration that could send data to the cloud. One test tool was allowed to use a free cloud model for its heaviest calls, and only because everything it handled was public market data.
- A written rule staff can follow. "Client names and personal details go only into the private assistant" is easier to follow than a 20-page policy.
- Logs you read. Every call in our lab was traced to a named client key, which is how we found an agent that had been refused 20 times in five days without anyone noticing.
For example, an accounting practice might give all staff Copilot for email and internal documents after a permissions clean-up, give two people a ChatGPT business seat for research, and run tax-return summarisation on a private model inside the office network so client data never leaves it.
Using AI at work? We help you set AI rules people will follow, check what data your tools can reach, and meet your privacy obligations.
Before you sign anything: a short checklist
- Check: your vendor’s data page confirms no training on your data for the exact plan you’re buying.
- Check: you have, in writing, where prompts are stored and where they’re processed.
- Check: retention for prompts and responses is set, and you know how to export or delete them.
- Check: for Copilot, a sample of SharePoint sites shared with "Everyone" or by anyone-links has been reviewed and fixed.
- Check: for private AI, someone is named as the operator, with monitoring, patching and backups in place.
- Check: staff have a one-page AI use rule that says which data goes where.
Related services: local and private AI, Copilot readiness, AI security and governance and AI consulting.
Key takeaways
- Decide per type of data, not per organisation: most end up using two or three of these options.
- Business ChatGPT plans and Copilot don’t train on your data by default; personal ChatGPT accounts do unless the user opts out.
- Australian residency for cloud AI is improving but differs by plan and feature, so get storage and processing locations in writing.
- Copilot shows people what they can already open, so fix permissions before you license it.
- Private AI keeps data on your hardware and costs nothing per token, but in our lab it meant queueing, smaller models and real operations work.
- Route by data classification through one gateway, with a short written rule and logs someone reads.
Sources
- OpenAI, Business data privacy, security and compliance: https://openai.com/business-data/
- OpenAI Help Center, data controls and model training for ChatGPT: https://help.openai.com/en/articles/7730893-data-controls-faq
- OpenAI Help Center, data residency for ChatGPT: https://help.openai.com/en/articles/9903489
- OpenAI Help Center, how your data is used to improve model performance: https://help.openai.com/en/articles/5722486
- OpenAI API documentation, data controls and data residency: https://developers.openai.com/api/docs/guides/your-data
- Microsoft Learn, Data, privacy and security for Microsoft Copilot: https://learn.microsoft.com/en-us/copilot/microsoft-365/microsoft-365-copilot-privacy
- Microsoft Learn, Enterprise data protection in Microsoft Copilot and Copilot Chat: https://learn.microsoft.com/en-us/copilot/microsoft-365/enterprise-data-protection
- Microsoft Learn, Overview of Microsoft Copilot Chat: https://learn.microsoft.com/en-us/copilot/overview
- Microsoft 365 Blog, in-country data processing for Microsoft 365 Copilot (with April 2026 update): https://www.microsoft.com/en-us/microsoft-365/blog/2025/11/04/microsoft-offers-in-country-data-processing-to-15-countries-to-strengthen-sovereign-controls-for-microsoft-365-copilot/
- OAIC, Guidance on privacy and the use of commercially available AI products: https://www.oaic.gov.au/privacy/privacy-guidance-for-organisations-and-government-agencies/guidance-on-privacy-and-the-use-of-commercially-available-ai-products
- NVIDIA, DGX Spark specifications: https://www.nvidia.com/en-us/products/workstations/dgx-spark/
- Catharsis lab series, Running local AI in production: /blog/running-local-ai-in-production-series-guide/
Want a second pair of eyes?
Catharsis Cyber Security is a Sydney cyber security company, certified to ISO/IEC 27001:2022. We help organisations protect, test and run their technology, and explain it in plain English.
