Staging batches

The Staging API import process is built around the concept of staging batches each of which is linked to a single client. Each staging batch will contain one or more staging documents and each staging document can contain zero or more rows.
  

Batches can be viewed either from the individual Client or from the main batches view.

A customer can take different approaches on sending batches. They can either

• Send batches, each containing a single document OR
• Send batches, each containing multiple documents

It is up to the customer requirements which approach suites better. 

Batches are processed by Aico one at a time and depending on the staging client configuration the processing behaviour can be different. For example it can be configured that if a single document in a batch is invalid, then none of the documents in the batch will be imported into Aico. Alternatively, you could configure Aico to import all valid and invalid documents in a batch and require the invalid documents to be manually corrected using the Aico user interface. 

A staging batch can go through the following statuses:

•  Open - the batch is open allowing it to accept data through the API.  It is not mandatory to send all the data in the batch at the same time. The customer can continue to send data to the batch until it is closed.

• Closed - the batch will no longer accept any new data through the API.  The staging service will only process batches which are in the closed status.
• Running - the staging service is processing the staging documents and turning them into Aico documents.  It first creates the Aico documents in memory and validates them (against the staging import validations described above). 
  • If any document errors i.e. can't be successfully saved then the whole batch errors and NO Aico documents are created.  
  • If all documents are valid then ALL the Aico documents are created and the batch is set to Completed. 
  • If any document is invalid then the Save invalid documents flag is considered.  
    • If that is True then ALL the Aico documents are created and the batch is set to Completed. If that is set to False then NO Aico documents are created. 
• The final status of a batch is Completed or Error. 
  • If the batch is Completed then this is the end of the process and nothing further will ever be done to the batch.  
  • If the batch is in Error then the Retry allowed is considered. 
    • If that flag is set to True then the user can correct the error and retry the original data sent in the batch. If that flag is set to False then the batch will remain in Error and a new batch will have to be sent in order to create those Aico documents.

Staging documents

As described above, the data in the batch is stored into the staging database and a staging document is created for each document in the batch. This process continues until the batch is closed meaning it will not accept any further documents. Note a staging document is NOT an Aico document and at this point the staging document has no status shown in the Status as “None”.  

To view the staging documents in the batch, click on the batch and then the documents tab.  Here you will find the details of each document and its status.  Note that you can see the header data sent for the document however you cannot view the row data.

Once the batch moves to running, and the Aico documents are created in memory they are each given one of 3 statuses: 

Document statuses

• Valid - Document has been validated successfully against the staging import validation tasks.
• Invalid - Document has NOT been validated successfully against the staging import validation tasks.
• Error - Document could not be created as an Aico document

Aico Document import process

Aico staging import process can be controlled in a few different ways using the client configuration.

• Invalid documents allowed - When checked, Aico documents will be created even if they do not validate successfully. Batch will be marked as Completed even if there are invalid documents.
• Retry allowed - Batches that go into Error, can be re-processed from the Aico Staging user interface.

Below you can see the staging import process and how the client configuration options affect the outcome.

Only validation

It is possible (through the API) to set the batch to only validate the staging documents. Where “Only validation” is set to true in the API call, then Aico will create the Aico documents in memory only and validate them.  It will then return the status of Valid or Invalid for each document but it will NOT save any of the Aico document into the Aico database.  This is useful for testing the data being sent particularly in PRODUCTION without the risk of the document being created and posted.

Frequency of staging service import

The staging service runs periodically based on the StagingServiceCycleDuration which is one of the global parameters.  In the example below it is set to run every 30 seconds. This means that it will process any batches that are closed and, once the documents in those batches are processed, it will wait a further 30 seconds and check again for any batches that are closed. If the service encounters an error it will stop for the StagingServiceCycleErrorDuration which, in the example below, is set to restart after 5 minutes.