Testing FHIR Bulk Data Export with Postman

This tutorial walks you through setting up Postman to test FHIR Bulk Data Export APIs using OAuth 2.0 client credentials flow in the Preview environment. You'll learn the complete bulk export workflow including initiating exports, polling for completion, and downloading result files.

Important: This tutorial uses the Preview environment with test data. Do not use these techniques with production environments containing real patient health information.

Prerequisites

• Postman installed
• Request access to a CareConnect tenant in the Preview environment for testing
• Register your application as a System Access App with CareConnect, which will provide:
  • Client ID
  • Client Secret
  • Tenant ID
• Basic understanding of FHIR, OAuth 2.0, and asynchronous processing concepts
• Completion of the System Access tutorial is recommended

Getting Started

Bulk Data Export requires system-level access with specific bulk data scopes. Your CareConnect app registration must include bulk data export permissions to access the $export operations.

Overview

Bulk Data Export enables secure, asynchronous export of large FHIR datasets using the FHIR Bulk Data Access Implementation Guide. This tutorial covers:

1. Setting up a Postman environment with bulk data variables
2. Creating a collection with OAuth 2.0 client credentials authorization
3. The complete bulk export workflow:
   • CapabilityStatement validation
   • Group discovery and selection
   • Export initiation
   • Status polling
   • File download
4. Troubleshooting common bulk export issues

Step 1: Create a Postman Environment

Creating the Environment

1. Open Postman
2. Click the Environments tab in the left sidebar
3. Click Create Environment
4. Name your environment (e.g., "FHIR Bulk Export - Dev")

Adding Environment Variables

Add the following variables to your environment:

Variable Name	Type	Initial Value	Current Value
tenant_id	default	your-tenant-id	your-tenant-id
base_url	default	https://fhirtest.netsmartcloud.com/provider/system-access/v2	https://fhirtest.netsmartcloud.com/provider/system-access/v2
auth_base_url	default	https://fhirtest.netsmartcloud.com/auth	https://fhirtest.netsmartcloud.com/auth
client_id	default	your-client-id	your-client-id
bulk_data_scope	default	system/*.rs	system/*.rs
group_id	default		(optional - can use specific Group ID in URL instead)

Replace your-tenant-id with your actual tenant ID from your CareConnect app registration.

Adding Secrets

For client secret authentication:

1. In your environment, add a new variable
2. Set Variable Name to client_secret
3. Set Type to secret
4. Enter your client secret from your CareConnect app registration

Private Key JWT Authentication

CareConnect also supports private key JWT authentication, but Postman does not natively support this method. Private key JWT also requires your public key to be registered with CareConnect or accessible via a JWKS URI. If your app registration uses private key JWT:

• Register a separate application with client secret authentication for Postman testing
• For production private key JWT, ensure your JWKS URI is publicly accessible

Save and Select Environment

1. Select Save
2. Select your environment from the dropdown in the top-right corner

Step 2: Create a Collection with OAuth 2.0 Authorization

Creating the Collection

1. Select Collections in the left sidebar
2. Select Create Collection
3. Name it "FHIR Bulk Data Export"
4. Add a description: "Collection for testing FHIR Bulk Data Export APIs with asynchronous workflow"

Adding Collection Variables

Add variables for temporary URLs generated during the export workflow:

1. In your collection, select the Variables tab
2. Add the following variables:

Variable Name	Initial Value	Current Value
export_status_url
export_output_url

3. Select Save

Configuring OAuth 2.0 Authorization

1. In your collection, select the Authorization tab
2. Set Type to OAuth 2.0
3. Configure the following settings:

Field	Value
Token Name	CareConnect Bulk Export Token
Grant Type	Client Credentials
Access Token URL	{{auth_base_url}}/:tenant_id/oauth2/v1/token
Client ID	{{client_id}}
Client Secret	{{client_secret}}
Scope	{{bulk_data_scope}}
Client Authentication	Send as Basic Auth header

Step 3: Create API Requests

Request 1: CapabilityStatement

Verify the server supports bulk data export operations.

1. In your collection, select Add request
2. Name: "Get CapabilityStatement"
3. Method: GET
4. URL: {{base_url}}/:tenant_id/metadata
5. Authorization tab: Set Type to No Auth
6. Params tab:
   • Path Variables: Set tenant_id to your tenant ID or {{tenant_id}} to use environment variable
7. Select Save

Test the request:

• Select Send
• Look for $export operations in the CapabilityStatement
• Verify Group resource supports $export operation

Request 2: Get Access Token

Before making authenticated requests, obtain an access token:

1. Go to your collection's Authorization tab
2. Select Get New Access Token
3. Postman will exchange your client credentials for an access token
4. Select Use Token to apply it to your collection

Request 3: Group Search

Find available Groups for export.

1. Add a new request: "Group Search"
2. Method: GET
3. URL: {{base_url}}/:tenant_id/Group
4. Authorization tab: Set Type to Inherit auth from parent
5. Params tab:
   • Path Variables: Set tenant_id to your tenant ID or {{tenant_id}} to use environment variable
   • Query Params: Add query parameters:
     • Key: _count, Value: 10
6. Select Save

Test the request:

• Select Send
• Look for Groups in the response
• Copy a Group ID from entry[].resource.id for use in the export request's path parameter
• Optionally update your group_id environment variable for reuse

Request 4: Initiate Group Export

Start the bulk export process for a specific Group. The Bulk Data Export specification supports both GET and POST methods:

GET Request

1. Add a new request: "Initiate Group Export (GET)"
2. Method: GET
3. URL: {{base_url}}/:tenant_id/Group/:group_id/$export
4. Authorization tab: Set Type to Inherit auth from parent
5. Headers tab: Add required headers:
   • Key: Accept, Value: application/fhir+json
   • Key: Prefer, Value: respond-async
6. Params tab:
   • Path Variables:
     • Set tenant_id to your tenant ID or {{tenant_id}} to use environment variable
     • Set group_id to a specific Group ID or {{group_id}} to use environment variable
   • Query Params (optional):
     • Key: _type, Value: Patient,Observation,Condition (specify resource types)
     • Key: _since, Value: 2023-01-01T00:00:00Z (export data since date)
     • Key: _outputFormat, Value: application/fhir+ndjson (or ndjson or application/ndjson)
7. Select Save

POST Request

1. Add a new request: "Initiate Group Export (POST)"
2. Method: POST
3. URL: {{base_url}}/:tenant_id/Group/:group_id/$export
4. Authorization tab: Set Type to Inherit auth from parent
5. Headers tab: Add required headers:
   • Key: Accept, Value: application/fhir+json
   • Key: Prefer, Value: respond-async
   • Key: Content-Type, Value: application/fhir+json
6. Params tab:
   • Path Variables:
     • Set tenant_id to your tenant ID or {{tenant_id}} to use environment variable
     • Set group_id to a specific Group ID or {{group_id}} to use environment variable
7. Body tab: Select raw and JSON, then add (optional):

{
  "resourceType": "Parameters",
  "parameter": [
    {
      "name": "_type",
      "valueString": "Patient,Observation,Condition"
    },
    {
      "name": "_since",
      "valueInstant": "2023-01-01T00:00:00Z"
    },
    {
      "name": "_outputFormat",
      "valueString": "application/fhir+ndjson"
    }
  ]
}

8. Select Save

Test the request:

• In the Params tab under Path Variables, set group_id to either:
  • A specific Group ID (e.g., group-123)
  • {{group_id}} to use your environment variable
• Select Send
• Expect a 202 Accepted response
• Copy the Content-Location header value
• Update your export_status_url collection variable with this URL

Export Parameters

• _type: Comma-separated list of resource types to export (optional)
• _since: Only export resources modified since this date (optional)
• _outputFormat: Format for exported files (optional, defaults to application/fhir+ndjson)
  • Supported formats: application/fhir+ndjson or ndjson
• _typeFilter: Apply search parameters to specific resource types (optional)

Request 5: Check Export Status

Poll the export status until completion.

1. Add a new request: "Check Export Status"
2. Method: GET
3. URL: {{export_status_url}}
4. Authorization tab: Set Type to Inherit auth from parent
5. Select Save

Test the request:

• Ensure export_status_url collection variable is set
• Select Send
• In Progress: Expect 202 Accepted with X-Progress and Retry-After headers
• Complete: Expect 200 OK with export manifest
• Error: Expect 4xx/5xx with error details

Polling Considerations

• Use the Retry-After header value to determine when to poll again (recommended wait time in seconds)
• If no Retry-After header is present, wait 1-2 minutes between polls
• Large exports may take 10-30 minutes or longer
• Check the X-Progress header for status and completion percentage

Request 6: Download Export Files

Download the exported NDJSON files.

1. Add a new request: "Download Export File"
2. Method: GET
3. URL: {{export_output_url}}
4. Authorization tab: Set Type to Inherit auth from parent
5. Select Save

Setup for file download:

• When export completes (status 200), copy a file URL from the output array
• Update your export_output_url collection variable
• The response will be NDJSON format (newline-delimited JSON)

Request 7: Cancel/Delete Export (Optional)

Cancel an in-progress export or clean up a completed export job.

1. Add a new request: "Cancel/Delete Export"
2. Method: DELETE
3. URL: {{export_status_url}}
4. Authorization tab: Set Type to Inherit auth from parent
5. Select Save

Test the request:

• Cancel in-progress export: Use when you need to terminate an active export
• Clean up completed export: Use after downloading files to tell the server it can delete the export data
• Expect 202 Accepted response for cancellation or cleanup
• The export job and associated files will be removed from the server

Step 4: Complete Bulk Export Workflow

Recommended Workflow Order

1. Get CapabilityStatement - Verify bulk export support
2. Obtain access token - Complete client credentials flow
3. Search for Groups - Find available Groups for export
4. Update group_id variable - Set the Group ID to export
5. Initiate Group Export - Start the export process
6. Update export_status_url - Save the status polling URL
7. Poll Export Status - Check status until completion
8. Download Export Files - Retrieve the exported data

Export Status Responses

In Progress (202 Accepted):

HTTP/1.1 202 Accepted
X-Progress: Running (45% complete)
Retry-After: 120

(empty response body)

Other X-Progress examples:

• X-Progress: Queued (0% complete) - Export is queued but not yet started
• X-Progress: Running (25% complete) - Export is actively processing
• X-Progress: Finalizing (98% complete) - Export is in final stages (>=95%)

Complete (200 OK):

{
  "transactionTime": "2024-01-15T10:30:00Z",
  "request": "https://fhirtest.netsmartcloud.com/provider/system-access/v2/tenant123/Group/group456/$export",
  "requiresAccessToken": true,
  "output": [
    {
      "type": "Patient",
      "url": "https://fhirtest.netsmartcloud.com/bulk-export/files/patients.ndjson"
    },
    {
      "type": "Observation", 
      "url": "https://fhirtest.netsmartcloud.com/bulk-export/files/observations.ndjson"
    }
  ],
  "error": []
}

Error (4xx/5xx):

{
  "resourceType": "OperationOutcome",
  "issue": [
    {
      "severity": "error",
      "code": "processing",
      "diagnostics": "Export failed due to insufficient permissions"
    }
  ]
}

Troubleshooting Common Issues

Authentication Issues

Problem: "Invalid scope" error during token request

• Solution: Verify your app registration includes bulk data export scopes
• Try system/*.rs or specific resource scopes like system/Patient.rs
• Ensure you're using system-level scopes, not patient-level

Problem: 403 Forbidden on export initiation

• Solution: Verify your access token has bulk export permissions
• Check if your app registration supports the $export operation
• Confirm the Group ID exists and is accessible

Problem: "Invalid client" error with private key JWT apps

• Solution: Postman doesn't support private key JWT authentication
• Private key JWT requires public key registration or accessible JWKS URI
• Register a separate application with client secret authentication for Postman testing

Problem: "HAPI-0333: Access denied by rule: Request is not authorized for this Tenant" error

• Solution: This error can occur due to tenant configuration OR insufficient authorization:
  • Tenant Issues: Verify your tenant_id path variable is set correctly in each request
  • Tenant Issues: Ensure the tenant ID matches your CareConnect app registration
  • Tenant Issues: Check that you're using the correct tenant ID for your environment (Preview vs Production)
  • Authorization Issues: Verify your access token has the required scopes for the resource or operation
  • Authorization Issues: Confirm your app registration includes permissions for the specific FHIR resources and operations
  • Authorization Issues: Check that your requested scopes were actually granted during the authorization process

Export Process Issues

Problem: 404 Not Found on Group export

• Solution: Verify the Group ID exists using Group search
• Check that the Group resource supports $export operation
• Ensure your group_id environment variable is set correctly

Problem: Export stuck in "In Progress" status

• Solution: Large exports can take 30+ minutes but should show incrementing percentages
• Check X-Progress header - percentage should increase over time even for long exports
• If percentage remains static for extended periods, verify server hasn't encountered processing errors
• Consider using _type parameter to limit resource types

Problem: Empty export results

• Solution: Verify the Group contains members
• Check if _since parameter is excluding all data
• Confirm the Group has associated resources to export

File Download Issues

Problem: 401 Unauthorized on file download

• Solution: Ensure your access token is still valid
• Some servers require fresh tokens for file downloads
• Check if requiresAccessToken is true in export manifest

Problem: Large file download timeouts

• Solution: Increase Postman timeout settings
• Consider downloading files programmatically for large datasets
• Use streaming download for files over 100MB

Network and Performance Issues

Problem: Slow export processing

• Solution: Use _type parameter to limit resource types
• Add _since parameter to export only recent data
• Consider smaller Group sizes for faster processing

Problem: Rate limiting errors

• Solution: Reduce polling frequency to every 2-3 minutes
• Implement exponential backoff for status checks
• Avoid concurrent export requests

Best Practices

Export Optimization

• Use selective exports: Specify _type parameter to export only needed resources
• Incremental exports: Use _since parameter for regular data synchronization
• Monitor progress: Check X-Progress header to estimate completion time
• Batch processing: Process downloaded NDJSON files in chunks

Security

• Secure file handling: Downloaded files contain PHI - handle securely
• Token management: Refresh tokens before they expire during long exports
• Access logging: Monitor who initiates and downloads bulk exports
• Data retention: Delete downloaded files after processing per your data governance policies

Performance

• Optimal polling: Use Retry-After header value for polling timing, fallback to 1-2 minutes if not present
• Concurrent limits: Avoid multiple simultaneous exports per tenant
• Resource selection: Export only necessary resource types to reduce processing time
• Network considerations: Large files may require stable, high-bandwidth connections

Next Steps

Once you've mastered bulk data export, consider:

• System-level Export: Test $export at the system level for all data
• Patient-level Export: Test Patient/$export for patient-specific exports
• Advanced Filtering: Use _typeFilter for complex export criteria
• Automated Workflows: Script the complete export process for regular data synchronization
• Error Handling: Implement robust error handling and retry logic
• Data Processing: Build pipelines to process exported NDJSON files

Additional Resources

• FHIR Bulk Data Access Implementation Guide
• SMART App Launch Implementation Guide
• FHIR R4 Specification
• NDJSON Format Specification
• Postman OAuth 2.0 Documentation