Getting data from where it lives to where you need it is a fundamental challenge. Your most valuable data—from SQL databases, ERP systems, and local file servers—often sits behind a protective firewall. Palantir Foundry solves this by using a secure, agent-based architecture to bridge the gap between these on-premises systems and its cloud environment.

Think of a Foundry agent as a highly trusted, security-cleared courier stationed inside your organization's secure facility. It never accepts unsolicited packages from the outside. Instead, it periodically checks a secure external mailbox for instructions. When it receives a request, it retrieves the specified package (your data), seals it, and sends it out through a pre-approved, outbound-only channel. This ensures the gates to your facility are never left open.

This courier, or agent, is a small piece of software installed within your network. It's responsible for all communication with Foundry. Crucially, this communication is always unidirectional—the agent initiates all connections, pulling instructions from Foundry and then pushing data out. Foundry never directly reaches into your network, which keeps your firewall intact and your systems secure. Each agent is authenticated using restricted tokens that grant it permission to perform only specific, pre-approved tasks, further minimizing the security risk.

Within Foundry, the connection to an external system is defined by two key metadata objects: the 'Source' and the 'Sync'. It's important to understand the difference. A Source is the blueprint, while a Sync is the action.

The Source contains the configuration details needed to connect to a system like a PostgreSQL database or an SFTP server. It holds the address, the driver type, and credentials, but it doesn't contain any actual data. It's simply a static definition of a connection point.

A Sync, on the other hand, is the execution unit. It references a Source to know where to connect, and then specifies which data to pull (e.g., a specific table or file), where to put it in Foundry (a specific dataset), and how often to run (the schedule). You can have many Syncs pointing to a single Source, each pulling different data into different Foundry datasets.

Let's walk through the process. First, an administrator configures a Source in Foundry, defining the target database. Next, a data engineer creates a Sync that tells the agent to query a specific table from that Source every hour and ingest the results into a versioned Foundry dataset.

One of the most powerful features this architecture enables is Source Exploration. A user in the Foundry UI can browse the schemas of an on-premises database without having direct network access to it. When the user clicks 'explore', Foundry sends a request to the agent. The agent then runs the necessary queries on the database locally, gets the schema (e.g., table and column names), and sends just that metadata back to Foundry. This allows for data discovery and pipeline building without ever exposing the database directly to the internet.

For mission-critical pipelines, you can't rely on a single agent. Foundry supports high-availability configurations by allowing you to group multiple agents into a resource group. When a sync is scheduled to run, Foundry assigns the job to any available agent in that group. If one agent machine goes down for maintenance or fails, another simply picks up the next scheduled sync, ensuring data continues to flow without interruption.

Let's check your understanding of these core components.

This agent-based architecture provides a secure and scalable foundation, allowing Foundry to integrate deeply with existing enterprise systems while respecting strict security boundaries.