Skip to main content

Shopify

This page contains the setup guide and reference information for the Shopify source connector.

Setup guide​

This connector supports the OAuth2.0 and API Password (for private applications) athentication methods.

Connect using OAuth2.0 (for Cloud users only)​

  1. Click Authenticate your Shopify account to start the autentication.
  2. Click Install to install the Airbyte application.
  3. Log in to your account, if you are not already logged in.
  4. Select the store you want to sync and review the consent.
  5. Click on Install to finish the Installation.
  6. Reveiew the Shop Name field for the chosen store for a sync.
  7. Set the Start Date as the starting point for your data replication. Any data created before this date will not be synced.
  8. Click Test and Save to finish the source set up.

Connect using API PASSWORD option (for OSS users)​

  1. Name your source.
  2. Enter your Store name. You can find this in your URL when logged in to Shopify or within the Store details section of your Settings.
  3. Enter your Admin API access token. To set up the access token, you will need to set up a custom application. See instructions below on creating a custom app.
  4. Click Set up source

Creating a Custom App​

Authentication to the Shopify API requies a custom application. Follow these instructions to create a custom app and find your Admin API Access Token.

  1. Navigate to Settings > App and sales channels > Develop apps > Create an app
  2. Name your new app
  3. Select Configure Admin API scopes
  4. Tick all the scopes prefixed with read_ (e.g. read_locations,read_price_rules, etc ) and save. See below for the full list of scopes to allow.
  5. Click Install app to give this app access to your data.
  6. Once installed, go to API Credentials to copy the Admin API Access Token.

You're ready to set up Shopify in Airbyte!

Scopes Required for Custom App​

Add the following scopes to your custom app to ensure Airbyte can sync all available data. To see a list of streams this source supports, see our full Shopify documentation.

  • read_analytics
  • read_assigned_fulfillment_orders
  • read_gdpr_data_request
  • read_locations
  • read_price_rules
  • read_product_listings
  • read_products
  • read_reports
  • read_resource_feedbacks
  • read_script_tags
  • read_shipping
  • read_locales
  • read_shopify_payments_accounts
  • read_shopify_payments_bank_accounts
  • read_shopify_payments_disputes
  • read_shopify_payments_payouts
  • read_content
  • read_themes
  • read_third_party_fulfillment_orders
  • read_translations
  • read_customers
  • read_discounts
  • read_draft_orders
  • read_fulfillments
  • read_gift_cards
  • read_inventory
  • read_legal_policies
  • read_marketing_events
  • read_merchant_managed_fulfillment_orders
  • read_online_store_pages
  • read_order_edits
  • read_orders

Supported sync modes​

The Shopify source supports both Full Refresh and Incremental syncs. You can choose if this connector will copy only the new or updated data, or all rows in the tables and columns you set up for replication, every time a sync is run.

This source can sync data for the Shopify REST API and the Shopify GraphQl API.

Troubleshooting tips​

  • If you faced with access errors while using the OAuth2.0, please make sure you've followed this Shopify Article to request the access to the Client's store first, once the access is granted, you should be able to proceed with OAuth2.0 authentication method.
  • Check out common troubleshooting issues for the Shopify source connector on our Airbyte Forum here.

Supported Streams​

This Source is capable of syncing the following core Streams:

Stream sync recommendations​

For better experience with Incremental Refresh the following is recommended:

  • Order Refunds, Order Risks, Transactions should be synced along with Orders stream.
  • Discount Codes should be synced along with Price Rules stream.

If child streams are synced alone from the parent stream - the full sync will take place, and the records are filtered out afterwards.

Capturing Deleted Records​

The connector captures deletions for records in the Articles, Blogs, CustomCollections, Orders, Pages, PriceRules and Products streams.

When a record is deleted, the connector outputs a record with the ID of that record and the deleted_at, deleted_message, and deleted_description fields filled out. No other fields are filled out for the deleted records.

Check the following Shopify documentation for more information about the Retrieving Deleted Records

Data type mapping​

Integration TypeAirbyte Type
stringstring
numbernumber
arrayarray
objectobject
booleanboolean

Features​

FeatureSupported?(Yes/No)
Full Refresh SyncYes
Incremental - Append SyncYes
NamespacesNo

Performance considerations​

Shopify has some rate limit restrictions. Typically, there should not be issues with throttling or exceeding the rate limits but in some edge cases, user can receive the warning message as follows:

"Caught retryable error '<some_error> or null' after <some_number> tries. Waiting <some_number> seconds then retrying..."

This is expected when the connector hits the 429 - Rate Limit Exceeded HTTP Error. With given error message the sync operation is still goes on, but will require more time to finish.

Changelog​

VersionDatePull RequestSubject
1.1.42023-10-1931599Base image migration: remove Dockerfile and use the python-connector-base image
1.1.32023-10-1731500Fixed the issue caused by the missing access token while setup the new source and not yet authenticated
1.1.22023-10-1331381Fixed the issue caused by the state presence while fetching the deleted events with pagination
1.1.12023-09-1830560Performance testing - include socat binary in docker image
1.1.02023-09-0730246Added ability to fetch destroyed records for Articles, Blogs, CustomCollections, Orders, Pages, PriceRules, Products
1.0.02023-08-1129361Migrate to the 2023-07 Shopify API Version
0.6.22023-08-0929302Handle the Internal Server Error when entity could be fetched
0.6.12023-08-0828291Allow shop field to accept *.myshopify.com shop names, updated OAuth Spec
0.6.02023-08-0228770Added Disputes stream
0.5.12023-07-1328700Improved error messages with more user-friendly description, refactored code
0.5.02023-06-1327732License Update: Elv2
0.4.02023-06-1327083Added CustomerSavedSearch, CustomerAddress and Countries streams
0.3.42023-05-1025961Added validation for shop in input configuration (accepts non-url-like inputs)
0.3.32023-04-1225110Fixed issue when cursor_field is "None", added missing properties to stream schemas, fixed access_scopes validation error
0.3.22023-02-2723473Fixed OOM / Memory leak issue for Airbyte Cloud
0.3.12023-01-1621461Added discount_applications to orders stream
0.3.02022-11-1619492Added support for graphql and add a graphql products stream
0.2.02022-10-2118298Updated API version to the 2022-10, make stream schemas backward cpmpatible
0.1.392022-10-1317962Added metafield streams; support for nested list streams
0.1.382022-10-1017777Fixed 404 for configured streams, fix missing cursor error for old records
0.1.372022-04-3012500Improve input configuration copy
0.1.362022-03-229850Added BalanceTransactions stream
0.1.352022-03-0710915Fixed a bug which caused full-refresh syncs of child REST entities configured for incremental
0.1.342022-03-0210794Minor specification re-order, fixed links in documentation
0.1.332022-02-1710419Fixed wrong field type for tax_exemptions for Abandoned_checkouts stream
0.1.322022-02-1810449Added tender_transactions stream
0.1.312022-02-0810175Fixed compatibility issues for legacy user config
0.1.302022-01-249648Added permission validation before sync
0.1.292022-01-209049Added shop_url to the record for all streams
0.1.282022-01-199591Implemented OAuth2.0 authentication method for Airbyte Cloud
0.1.272021-12-229049Updated connector fields title/description
0.1.262021-12-148597Fixed mismatched number of tables for base-normalization, increased performance of order_refunds stream
0.1.252021-12-028297Added Shop stream
0.1.242021-11-307783Reviewed and corrected schemas for all streams
0.1.232021-11-157973Added InventoryItems
0.1.222021-10-187101Added FulfillmentOrders, Fulfillments streams
0.1.212021-10-147382Fixed InventoryLevels primary key
0.1.202021-10-147063Added Location and InventoryLevels as streams
0.1.192021-10-116951Added support of OAuth 2.0 authorisation option
0.1.182021-09-216056Added pre_tax_price to the orders/line_items schema
0.1.172021-09-175244Created data type enforcer for converting prices into numbers
0.1.162021-09-095965Fixed the connector's performance for Incremental refresh
0.1.152021-09-025853Fixed amount type in order_refund schema
0.1.142021-09-025801Fixed line_items/discount allocations & duties parts of orders schema
0.1.132021-08-175470Fixed rate limits throttling
0.1.122021-08-095276Added status property to product schema
0.1.112021-07-234943Fixed products schema up to API 2021-07
0.1.102021-07-194830Fixed for streams json schemas, upgrade to API version 2021-07
0.1.92021-07-044472Incremental sync is now using updated_at instead of since_id by default
0.1.82021-06-294121Added draft orders stream
0.1.72021-06-264290Fixed the bug when limiting output records to 1 caused infinity loop
0.1.62021-06-244009Added pages, price rules and discount codes streams
0.1.52021-06-103973Added AIRBYTE_ENTRYPOINT for Kubernetes support
0.1.42021-06-093926New attributes to Orders schema
0.1.32021-06-083787Added Native Shopify Source Connector
Was this page helpful?