FRAMEWORK · PRIVACY-FIRST PROCUREMENT · 6 min
Privacy-First AI Adoption
Hospital teams routinely expose Protected Health Information during AI vendor demos and pilots. They do it inadvertently — uploading a "sample" chart, recording a "test" patient encounter, sending an internal SOP through a vendor sandbox — because the procurement choreography has not yet asked the privacy questions. This is the framework to use before the demo, not after it.
The six decisions that protect the buyer's compliance posture before any data touches a vendor system. Make these in writing, before the first demo.
Across Canadian provincial enforcement and HIPAA case law, the most common compliance failure is a Privacy Impact Assessment completed after deployment rather than before procurement hardens.
PIPEDA / CPPA amendments clarify that consent for AI training and AI-driven decision-making is distinct from consent for service delivery. Provincial regimes are catching up in parallel.
The decision: privacy-first or procurement-first?
The default procurement pattern in healthcare AI is to invite three vendors to demo, watch the demos with sample data the IT team supplies, run a four-week pilot with anonymized records, and finalize the contract afterwards. That sequence puts the buyer in the worst possible negotiating position: the privacy work is reactive, the vendor's defaults become the baseline, and the BAA gets signed under time pressure to keep the project moving.
Privacy-first procurement flips the sequence. The compliance posture is decided before the demo. The vendor list is filtered against that posture before any data is uploaded. The pilot is scoped to use only data that has been pre-approved for the chosen vendor's exact deployment region and retention policy. The procurement committee gets to compare apples to apples on the things that actually matter — residency, retention, training-data use, audit, breach response — and the vendors that cannot answer cleanly get filtered out before the demo even happens.
This is not a slower process. It is the same total work, redistributed earlier. The trade-off the buyer makes is "one or two extra hours of legal review before the demo" against "discovering the vendor's default audio retention is incompatible with our policy after we already showed them a sample of real patient audio."
The six pre-procurement decisions
- check1. Decide which deployment patterns are eligible. Cloud-only, customer-tenanted cloud, on-prem, or hybrid? The answer constrains the vendor list before the demo. For most Canadian hospitals operating under PHIPA, Quebec Law 25, or HIA, customer-tenanted or on-prem deployment options should be on the eligibility list; many U.S. cloud-only vendors will be ineligible.
- check2. Decide the residency boundary in writing. "PHI can leave the hospital network" / "PHI can leave the province" / "PHI must remain inside the building." Each draws a different vendor circle and a different contract clause. Write the boundary down before the demo or the boundary will be defined by whichever vendor demos first.
- check3. Decide the training-data posture. Will the buyer permit vendor model training on the hospital's audio, transcripts, or notes? If yes, under what consent and de-identification rules? If no, the contractual prohibition needs explicit language. The 2026 PIPEDA / CPPA amendments now make this consent distinct from service-delivery consent.
- check4. Decide audio and transcript retention. Default vendor retention policies vary — some delete audio after note generation, others retain it indefinitely for "quality." The buyer's retention policy should determine the contractual default, not the vendor's. The original audio is also a clinical-safety control for verifying suspicious transcripts; treat retention as a safety question, not just a privacy one.
- check5. Decide the audit and breach posture. What audit telemetry does the buyer require? What breach-notification timeline matches the strictest applicable provincial regime? These belong in the BAA before signing, not after a discovery period.
- check6. Run the Privacy Impact Assessment before the demo. Quebec Law 25 makes PIAs mandatory for tech processing personal information; Ontario's IPC strongly recommends them under PHIPA. The PIA is the artifact that demonstrates the rest of the compliance work is real — and it is most useful when it happens before the buyer commits emotionally to a specific vendor.
The privacy-first demo
With the six decisions documented, the vendor demo changes shape. The buyer does not bring sample PHI; the buyer brings the buyer's policy and asks the vendor to demonstrate compliance with it. Three concrete moves that change the demo from "watch the vendor's product" to "watch the vendor's compliance":
Ask the vendor to draw — physically, on a whiteboard — every place PHI moves through their system, every region it lands in, every subprocessor that touches it. If the vendor cannot draw this without consulting documentation, the data path is not under their day-to-day operational control.
Ask for the BAA before the demo, not as a follow-up. Read it. The vendors who treat the BAA as a sign-after-procurement detail are the vendors whose contractual residency commitments will be soft. The vendors who hand it over at the first conversation tend to be the ones who have already operationalized it.
Ask the vendor to show what an exportable customer-side audit-log artifact looks like for a recent customer. Not a marketing slide — the actual export format. The audit story is where contractual commitments meet operational reality; vendors who cannot show the artifact tend not to have operationalized the export.
The privacy-first pilot
The pilot is the highest-risk phase for inadvertent PHI exposure — real clinicians, real patients, real audio, often under time pressure to "just test something." Three safeguards that keep the pilot inside the compliance posture chosen during pre-procurement:
- checkPatient consent forms updated before the pilot starts. Patient consent for AI scribe use is distinct from generic clinical-recording consent, and Quebec Law 25 Section 12 adds the algorithmic-transparency disclosure requirement. The hospital's consent forms should reflect the specific vendor, the specific data path, and the specific automated-decision components.
- checkPilot scope tied to the eligibility decision. If the eligibility decision is "no PHI leaves the province," the pilot must run on customer-tenanted or on-prem infrastructure — even if the vendor offers a "convenience" sandbox in a different region. Pilot exceptions tend to harden into production defaults.
- checkStop conditions written into the pilot agreement. The conditions that pause or terminate the pilot — audit log gaps, breach, safety event, residency violation — should be in writing before clinicians start recording encounters. A pilot without written stop conditions is a pilot that drifts.
What this changes in vendor selection
Privacy-first procurement changes which vendors look strong on a shortlist. Vendors that already operationalize transparent BAAs, data-minimal defaults, and clear residency commitments climb the list — Nabla's no-audio-stored default, Freed's deleted-after-note-generation policy, Abridge's mature Trust Center. Vendors whose defaults shift quietly with each platform consolidation become harder to recommend — Commure Ambient's post-Augmedix product rationalization is a real procurement question.
For Canadian residency-bound buyers, the privacy-first lens also clarifies why on-prem alternatives are on the shortlist. The compliance work for an on-prem deployment is "describe the data path on our own infrastructure" — shorter, simpler, and easier to defend than the contractual residency choreography required to keep a U.S. cloud vendor compliant with Quebec Law 25.
Where this fits in the WalledCare directory
Use this framework as the front matter for the procurement process the WalledCare directory supports. Pair it with the Canadian compliance hub for the regulatory layer, the RFP-questions checklist for the procurement-document specifics, the safety reference for the clinical-safety lens, and the vendor side-by-side comparison to filter the shortlist before the first demo.