Skip to main content

Guide Catalog operations

Reading processed products back: status, polling and reconciliation.

In short

An accepted product is not a finished one. Read each product's status, poll politely with a limit, reconcile the fields that came back and keep your own systems authoritative for their data.

Published by Youzu · Editorial policy

Published
Reading time
8 min read

Key points

  • Check each product's status before you use it: acceptance is only the start.
  • Use a delay that grows between polls, and stop after a limit you choose.
  • Compare what came back with what you sent, field by field.
  • Keep your own system authoritative, and write back only what you accept.
In this guide

Accepted is not finished

Many product APIs answer a submission with a short acknowledgement and do the work afterwards. In HTTP terms, the request was accepted for processing, and the processing has not been completed.[1] For a catalog, that means a submitted product can be stored, queued, partly processed, finished or failed, and the first reply tells you only that it was received.

Plan for that gap. If your storefront shows a product the moment it is accepted, shoppers will sometimes see records that are still being processed. Give each product a state in your own system, and move it forward only when the status you read back says it can move.

Diagram

From accepted to read back

From accepted to read back4 stacked steps. Accepted: the data is stored and queued. Processing: steps run on each product. Finished, partial or failed: each product reports its own state. Read back and reconcile: the reader compares fields and accepts or corrects.AcceptedStored and queuedstatusProcessingSteps run on each productstatusFinished, partial or failedEach product reports its statereadRead back and reconcileCompare fields, accept or correct
A submission is accepted first and processed afterwards. The reader polls the status, then reconciles what came back. The state names are generic; take the real ones from the API reference.

Read the status, not the clock

Waiting a fixed time and hoping is the most common reading strategy and the least reliable. Processing time varies with the size of the batch, the steps involved and the load at that moment. Read the status of each product instead, and act on what it says.

A generic set of states and what a reader does with each
StateMeaningWhat to do
AcceptedStored, not yet processedKeep the identifier and wait
ProcessingSteps are runningPoll again later; do not publish
FinishedAll steps are completeRead the record and reconcile it
PartialSome steps are completeUse what is finished; keep the rest pending
FailedA step stopped or the data was rejectedRead the reason, correct the data, send it again

The exact names of the states belong to the API you use, so take them from its reference rather than from this table. Write your own state model first, then map the API's states onto it, so that a change in the API's wording does not change what your storefront does.[2]

Polling without hammering

Polling means asking again. Do it politely. Start with a short delay, lengthen it after each attempt and stop after a limit you choose, then mark the product as needing attention instead of asking forever. Services protect themselves with request limits, and a loop that asks every second for 10,000 products will meet one.

  1. Read in batches with a filter, such as the products still processing, instead of one product at a time.
  2. Lengthen the delay between rounds: for example 10 seconds, then 30, then a minute.
  3. Stop after a limit, and put what remains into a visible queue for a person.
  4. Respect a request limit if the service reports one, and slow down instead of retrying at once.

Read the filters the API offers before you design the loop. Being able to ask only for products in a given state, in a given category or changed since a given time removes most of the load, and most of the code.

Reconcile what came back

A finished product is not automatically correct. Reconciliation means comparing what you sent with what you read back and deciding what to accept. Do it field by field: the identifier should match, the category should be one of yours, typed values should keep their types and an empty value should stay empty unless someone has filled it.

A reconciliation check for one product
FieldSentRead backDecision
IdentifierSame codeSame codeAccept
ColorOliveOliveAccept
SizeMMAccept
Material(empty)(empty)Keep empty
CategoryJacketsOuterwearReview: not your category

Decide in advance which fields you will accept without a look and which go to a person. Count what you review: the share of products with at least one field that needed a decision tells you whether the process is helping or only moving the work.

A worked example: 40 products in one batch

Suppose a team sends a batch of 40 products and gets 40 acknowledgements straight away. Nothing has been processed yet. The reading loop starts by asking for the products still in progress and records what it sees at each round.

One synthetic batch of 40 products over 4 polling rounds
RoundProcessingFinishedPartial or failed
1 (after 10 seconds)4000
2 (after 30 seconds)22171
3 (after 1 minute)4333
4 (after 2 minutes)0364

The numbers are round values invented for this page to show the shape of a run; they are not a measurement of any service. By round 4 the loop has stopped asking, 36 products are ready to reconcile and 4 need a person: the reader opens each one, reads the reason and decides whether to correct the data and send it again or to leave the product out.

Notice what the reader never did: it never published the 40 products on acceptance, never waited a fixed 10 minutes and never asked about a product that had already finished. Those 3 habits are the difference between a loop that scales and one that has to be rewritten the first time a large batch arrives.

Errors, retries and idempotence

Not every problem is a failed product. A request can time out, a connection can drop and a service can answer that you are asking too often. Treat these as reasons to retry the read, not as news about the product, and keep the retry separate from the state of the product itself.

When you send a product again, make the send safe to repeat. Sending the same product twice should end with one product, not 2, so identify each product by a stable code of your own and send that code every time. A reader that can retry without fear is simpler than one that has to remember which sends succeeded.

  • Retry reads after a timeout with a growing delay, and keep a counter of attempts.
  • Send each product with a stable code of your own so a repeat replaces instead of duplicating.
  • Record the reason a product failed, in words a catalog editor can act on.
  • Show failed products in a queue a person can see, with an owner.

Keeping watch

A reading loop that runs unattended should report on itself. Track how many products wait in each state, how long the oldest one has waited and how many ended partial or failed in the last day. A rising count of the oldest waiting product is the earliest sign that something upstream is wrong.

Review a few finished products by hand each week, even when the numbers look fine. Numbers show that the loop ran; a person reading a record shows whether what came back is what you wanted.

Which system stays authoritative

Reading data back does not make the service the owner of your catalog. Your PIM, your ERP and your storefront stay authoritative for their own data, and the processed records are an input that you accept, edit or ignore. Write back only what you have accepted, through the paths your own systems provide.

A good test is to ask what happens if the service is unavailable for a day. Your storefront should keep serving the records it already has, and the reading loop should resume where it stopped. If it cannot, the service has become a dependency your design never chose.

The glossary entry on asynchronous ingestion defines the idea in a paragraph, and the guide on marketplace catalog management shows where this loop sits among the other jobs.

Boundaries

What Youzu does not do

  • Youzu does not replace your ERP, search or storefront; those systems stay authoritative for their own data. It can replace your PIM, or work alongside it.
  • Youzu publishes no accuracy figures. Measure accuracy on your own sample against your own labels, with critical errors reported separately.
  • Youzu does not promise an onboarding time in advance. We size the timeline with you after seeing a sample.
How an evaluation runs

Checklist

Reading processed products back: checklist

10 checks across states, polling, reconciliation and ownership, with an owner column and an empty status column.

Download the checklist (CSV)

A CSV file of 10 rows, named youzu-reading-processed-products-back-checklist.csv. No form to fill in first.

Reading processed products back: checklist, scrollable
AreaCheckOwnerStatus
StatesYour own state model is written downEngineering
StatesEach API state maps onto your own statesEngineering
PollingDelays lengthen after each roundEngineering
PollingA limit stops the loop and queues the product for a personEngineering
PollingReads use filters instead of one product at a timeEngineering
ReconcileIdentifier, category and typed values are comparedCatalog operations
ReconcileEmpty values stay empty until someone fills themCatalog operations
ReconcileThe share of products needing a decision is countedCatalog operations
OwnershipYour own systems stay authoritativeArchitecture
OwnershipThe storefront keeps working if the service is unavailableArchitecture

FAQ

Questions buyers ask

Short answers to the questions teams ask about this topic.

No. Ingestion is asynchronous: an accepted request does not mean enrichment, indexing or deduplication has finished, and each product shows its processing status. Read the status before you publish a product, and treat anything without a finished status as work in progress.

Processed products, variants, offers, moderation state and processing status can be read back through the API with filters. Use the filters to ask only for what you need, such as the products still processing, instead of reading the whole catalog each time.

Changed products can be re-sent at any time; the same SKU identifies a later version of an offer. Send the corrected product, then read its status again before you rely on the new version, and keep your own stable code on every send so that a repeat replaces the earlier one.

You do not need a pre-built connector: send files or call the API. Choose whichever fits your systems, and keep the reading loop in your own code so that you control how often it asks and what it does with each state.

Youzu works alongside your existing PIM, ERP, search and storefront; those systems stay authoritative for their own data. Treat processed records as input that you accept, edit or ignore, and write back only what you have accepted through your own systems' paths.

Last reviewed

Sources

  1. RFC 9110: HTTP Semantics, 15.3.3 202 Accepted, IETF. rfc-editor.org/rfc/rfc9110. Accessed . Back

  2. OpenAPI Specification, OpenAPI Initiative. spec.openapis.org/oas/latest.html. Accessed . Back

About this guide

Published by Youzu · Editorial policy

Published

Every guide defines its terms, works through one example and cites the public sources it relies on, with the date each source was checked.

Guides explain methods. They do not rank Youzu above other approaches, and an example is never presented as a customer result.

More in all guides and the catalog glossary.

Next step

Automate Commerce. Make shopping personal.

Bring your catalog, listing flow or shopping idea. Let's find the right place to start.

Book a demo

Tell us about your catalog and we will get back to you.