Discuss.FOLIO.org is no longer used. This is a static snapshot of the website as of February 14, 2023.

RTAC (RTAC | Holdings) performance improvement - Technical Designs and Decisions - FOLIO Wiki

system
24 Jul '20

This page proposes several options of RTAC performance improvement, which relates to MODRTAC-38.

Background (how RTAC logic works without alteration)

  1. Go to /inventory/instances/<instance_id> to verify such instance exists
  2. Go to /holdings-storage/holdings and retrieve all holdings where instanceId matches requested Id
  3. For each holdings record go to /inventory/items and retrieve all items where holdingsRecordId matches holding Id.
  4. For each item go to /circulation/loans and retrieve all loans where itemId matches item Id and status equals “Open”

See getRtacById method for details.

After running performance testing on RTAC (see details in report) it was discovered that the time it performs isn’t appropriate, especially as for EDS UI. Therefore the time of processing RTAC request should be reduced.

O-complexity for REST requests (all of them are supposed to go through OKAPI and other ): 

Reduce calls by using CQL "or" statement

The first option is based on CQL “or” statement usage. The idea is to retrieve more items and loans with a minimum number of calls instead of making a request for each holdings’ and item’s Ids accordingly in a loop. So, the initial algorithm might transform as follows:

  1. Go to /inventory/instances/<instance_id> to verify such instance exists
  2. Go to /holdings-storage/holdings and retrieve all holdings where instanceId matches requested Id
  3. Gather all holdings Ids into CQL query separated by “OR” statement and retrieve all items where holdingsRecordId matches provided holdings Ids.
  4. Gather all item Ids into CQL query separated by “OR” statement and retrieve all loans where itemId matches provided item Ids and status equals “Open”.
ProsConsMinimal effort from the implementation sideImproves performance less than database viewDoesn’t require other modules modificationCQL query is limited by approximately 50 UUIDs per time ()

Create database view which supports API for RTAC in mod-inventory-storage

This view is supposed to take instance UUID as input parameter and return all necessary for RTAC response information. All filtering is done on SQL level, which proves to show great performance. Just one call to mod-inventory-storage API would be done. The algorithm might be next:

  1. GET /inventory/rtac-view?instanceId=<instance_id>
  2. From full representation Retrieve all items which are associated with requested instance Id.
  3. Check on items’ loans via REST calls
ProsConsSignificant performance improvement due database joins instead of REST callsSpecial data representation and corresponding endpoint in inventory-storage, which is needed (for now) only for mod_rtacOnly necessary data (fields) can be retrieved from mod-inventory-storageLoans are in different schema (diku_mod_circulation_storage), so 4th point should still be executed.Note:

Use of GraphQL

This approach is based on using GraphQL for fetching items information. GraphQL is basically a query language for API and it was implemented for FOLIO needs in mod-graphql module. The good thing about it is that one could specify CQL query in it, but without length limitations, as it goes to POST body, and build just the same set of fields which is needed. The next query could be utilized:

query ($requestedInstanceId:String) {
  instance_storage_instances(instanceId: $requestedInstanceId) {
    holdingsRecords2 {
      holdingsItems {
        id 
        effectiveLocationId
        effectiveCallNumberComponents {
          prefix
          callNumber
          suffix
        }
        status {
          name
        }
        permanentLoanTypeId
        temporaryLoanTypeId
        volume
        enumeration
        chronology
      }  
    }
  }
}

The high-level algorithm is presented below: 

  1. Go to /graphql by instance id
  2. Retrieve all items information which is associated with requested instance Id.
  3. Check on items’ loans via REST calls
ProsConsCorrect implementation allows to efficiently join entitiesCurrently in POC stageCurrent implementation implies REST calls to inventory-storage for every entityPerformance for current implementation is worse then CQL with OR

Preliminary decision

, such as: 

GET /inventory-storage/itemsByInstanceID?instanceId=<instance_id>
  • New API interface should return item information enriched with actual properties names where item object stores only property UUID (e.g. "effectiveLocation":{ "id": "fcd64ce1-6995-48f0-840e-89ffa2288371", "name": "Main Library" }). That is, result response should be similar to /inventory/items response. 
  • It also should be able to take multiple instance ids (up to 50) in order to cover requirements from  MODRTAC-34 - Getting issue details... STATUS .
  • Furthermore, an additional parameter (e.g. includeSuppressedFromDiscovery = {true, false}) should be introduced for successful  MODRTAC-35 - Getting issue details... STATUS  implementation.

This would benefit other situations, beyond just RTAC, where the data model hierarchy needs to be traversed - especially if domain boundaries are not crossed.

This would also reduce the problem to the following: instanceID → (instance → holdings → items) → loans where the traversal in parentheses is implemented at the database level and surfaced as a single API call.

This leaves potential 2 API calls (assuming interfaces which accept multiple UUIDs) and avoids crossing domain boundaries.


This is a companion discussion topic for the original entry at https://wiki.folio.org/display/DD/RTAC+%28RTAC+%7C+Holdings%29+performance+improvement