Data privacy and security concerns have increased in recent weeks as generative AI startups face public disputes over what their systems see and retain from customers. Canadian AI startup Cohere says it can assure enterprises their data remains private and off-limits to itself and cloud providers using Model Vault, a single-tenant inference product, and a technique called confidential computing.
Cohere’s solution runs the workload inside hardware that encrypts and isolates its memory, so neither the cloud provider nor Cohere itself can read what is inside. According to the company, that protection now covers the moment a model actually reads the prompt, not just the moments before and after.
How Model Vault Works
Model Vault is Cohere’s single-tenant inference platform, a dedicated deployment of Cohere’s models rather than a shared multi-tenant API. Confidential computing runs that deployment inside hardware that encrypts a workload’s memory and isolates it while the workload runs.
The hardware used by Model Vault’s new Encrypted tier pairs a confidential VM on the CPU side with Nvidia GPUs running in confidential computing mode. The CPU side runs on Intel TDX or AMD SEV-SNP, providing a hardware boundary behind the claim that Cohere cannot see the data.
Protection now spans both the CPU and the GPU, rather than stopping at the CPU the way most earlier confidential computing deployments did. The data path stays encrypted as it travels from the customer to the Confidential VM (CVM), remaining protected throughout the hardware path.
Verification and Attestation
Cohere’s director of serving inference, Manoj Govindassamy, noted that every inference returns an attestation report that lets customers verify the exact hardware, software, and security policies protecting their workload. This report allows customers to confirm that their data is secure and that Cohere cannot access it.
Cohere plans on making the full serving stack used in Model Vault open source, allowing independent auditors to validate that it doesn’t log, export, or leak data. The hardware and software ‘golden values’ are registered with Intel’s Trust Authority, providing an additional layer of verification.
Related Post: NTT DATA, Palo Alto Networks Forge AI Security Alliance
The competitive field for confidential inference is not new, with several vendors already selling some version of the promise that the operator cannot read the prompt. However, what separates Cohere’s Model Vault is that it applies confidential computing to Cohere’s own enterprise models, providing a vendor-signed attestation token that can be handed directly to an auditor.
Cohere’s launch of confidential computing support in Model Vault comes as the company signs its definitive merger agreement with Germany’s Aleph Alpha, a roughly $20 billion deal that creates dual headquarters in Toronto and Berlin. Aidan Gomez, Cohere’s CEO, pitched the combined company around trust and governability rather than raw model capability.
According to Govindassamy, “No one, including Cohere, the cloud provider or the cluster operator can see or access any customer data.” The data is only decrypted at the point where it must actually be processed inside the protected CPU and GPU execution environment, and the encrypted response is returned back to the user.
Cohere’s Model Vault and confidential computing system is designed to provide enterprises with a secure and trustworthy way to process their data. With the launch of confidential computing support, Cohere is providing a new level of protection for customer data, and setting a new standard for the industry. The company’s use of hardware-based encryption and isolation provides a strong foundation for secure data processing, and the attestation reports provide customers with the verification they need to trust the system.
Cohere’s Model Vault is now available with confidential computing support, and the company is providing this capability at no additional cost to its customers. The pricing remains the same as the standard Model Vault offering, making it an attractive option for enterprises looking for a secure and trustworthy way to process their data. As of today, Cohere’s Model Vault with confidential computing support is live, and customers can start using it to protect their sensitive data.
As the demand for confidential inference continues to grow, Cohere’s Model Vault is well-positioned to meet the needs of enterprises that require a high level of data protection. With its single-tenant inference platform and confidential computing capabilities, Model Vault provides a secure and trustworthy way for businesses to process their sensitive data.
Meeting the Needs of Regulated Industries
The ability to provide a vendor-signed attestation token that can be handed directly to an auditor provides an additional layer of assurance that the data is being protected.
