Overview

This app demonstrates how to pull data from Salesforce via REST API and drop it into an AWS RDS database on a schedule. It is pre-configured to pull Salesforce Account objects.

Use the Terraform found in the infrastructure folder to spin up the DB for this app, and deploy this app as a Docker container to AWS ECS.

Requirements

Database Server

You will need a database to run this app as it will drop all Salesforce record fetched into it. If you plan on using AWS RDS, this repo contains Terraform in the infrastructure/database folder that you can use to spin up an instance.

Salesforce Connected App

1. To authorize your app to the Salesforce API, you will need to generate a digital certificate that you will upload to the Salesforce connected app and sign the JWT using the certificates private key.
   1. Docs to Generate Certificate
2. Setup a Salesforce connected app
   1. Enable Enable OAuth Settings
   2. Enable Enable for Device Flow (will auto assign callback URL which you don't need for this type of app)
   3. Enable Use Digital Signatures
   4. Upload certificate created in previous step
   5. Add Selected OAuth Scopes
   6. After save:
      1. Go to App Manager >> [App Name] >> Manage
      2. Click Edit Policies
      3. Change Permitted Users to Admin approved users are pre-authorized
      4. Save
      5. On the app management page, scroll down to the Profiles section and add the profiles that are allowed to access the app

Configuration Variables

This app requires the following configuration variables that you can either pass in via environment variables or by adding a secrets.json file with the variables included to the project:

• ConnectionStrings:default - Default DB connection string
• Salesforce:ApiVersion - Salesforce API version to use (default is 51.0)
• Salesforce:ClientId - Salesforce connected app client Id
• Salesforce:User - Salesforce username
• Salesforce:AuthKey - Salesforce connected app OAuth certificate private key encoded as a base 64 string (you can use an online tool to base 64 encode the private key, or run base64 <privatekey> from a bash shell)

Initialize Database via Entity Framework

All configuration to spin up a database is in src/SalesforceDataCollector/Migrations. When dealing with a new database, just run the Update-Database from the Package Manager Console to apply the migrations to a new database on your server.

Mocking Salesforce Data

During development, if you need to quickly test this service against a large data-set, you can use mockaroo.com to create a randomized CSV file and then use dataloader.io or the Salesforce workbench to load all of the data at once into your developer account.