About the company

A leading health and security services firm providing multicultural health, security and logistics solutions from over 1,000 locations in 80+ countries.

Challenge

On its journey to further develop competitive products and services, the company wanted to design and build a modern data ingestion capability and revamp its legacy, extract transform and load, data storage and data consumption platforms. A primary goal was to deploy a decision engine by consolidating and streamlining content ingestion tools, processes and operations while helping to ensure data integrity and quality. Customer data was collected using multiple ingress protocols and ingested into a cloud-native enterprise data lake. The company was looking to design and integrate high-performance Application Programming Interfaces (APIs) for the ingestion and consumption of analytics and visualization tools, third-party integrations and web/mobile applications. The existing core apps included traveler management applications, a custom user provider for a single sign-on (SSO) system and an API–based platform being exposed for consumption.

Key requirements:

• Provide secure data upload mechanisms for end users using an SSO feature
• Provide a safe multilayered encrypted mechanism for customers to upload data
• Implement a configurable rules-based engine to the data ingestion pattern
• Process various file formats and record structures across data types and sizes
• Enable data availability for existing application consumption with minimal effort
• Serve as an enterprise platform and a single source to the employee database across the organization
• Gain scalability to new clients and initiatives planned by the organization during and beyond the pandemic
• Enable 24/7/365 data availability with privacy and security compliance

What we did

• Amazon Web Services (AWS) DynamoDB database service
• AWS Lambda data migration
• Data architecture and governance model
• Data ingress mechanism
• Serverless data ingestion pipeline
• Enterprise data lake

Solution

The firm was poised to embrace the public cloud for the first time with the help of Unisys Cloud, Applications & Infrastructure Solutions experts. AWS was the recommended data ingestion platform for its flexibility, reliability and scalability.

The solution architecture provided was categorized into the following sections:

• Customer onboarding and integration with SSO – API Gateway, AWS DynamoDB, Lambda
• Ingress mechanism: Secure API, Secure File Transfer Protocol (SFTP), Lambda, DynamoDB
• Serverless data ingestion pipeline: Lambda, DynamoDB, Glue
• Data storage: Elastic search, cloud-native data lake and application database consumption

AWS Lambda was used to extensively build several critical subsystems of the enterprise data ingestion platform. The fully managed serverless technologies delivered innovative capabilities for non-functional requirements and enabled the company’s engineering teams to increase focus on pressing business problems.

AWS DynamoDB was leveraged as a highly scalable, performant, available and easily restorable multi-region database service for multiple critical parts of the enterprise data ingestion platform. Our team provided software development lifecycle services, including architectural, design, development, test and deployment activities. Other services included:

• Designed and set up multiregion (U.S. Northeast and France) and multi-availability zone infrastructure to help ensure data security, availability and adherence to General Data Protection Regulation (GDPR) compliance
• Designed and delivered an AWS Lambda-based custom authorizer system for the AWS Managed SFTP service to provide secure file transfer capability; the custom authorizer can be integrated with an SSO customer database present in Okta
• Delivered an AWS API Gateway backed by AWS Lambda to build Representational State Transfer (REST) APIs needed for the solution; secure and scalable REST APIs were provided to perform various activities, including customer onboarding and verification
• Used AWS Lambda and AWS Glue to build data ingestion capabilities into the AWS Relational Database Service (RDS) database and evaluated the RDS proxy
• Used AWS Lambda to add extra application-level security with a robust public-private key mechanism and an in-place decryption system within the data pipeline mechanism
• At the heart of the data ingestion platform lies a complex state machine based on business rules that are changing continuously; Unisys implemented a complex, asynchronous and configurable multistep business rules engine carefully orchestrated using multiple AWS Lambdas
• Provided a scalable and reliable application database to be consumed by existing applications through API platforms by migrating the legacy Structured Query Language server

Result and outcomes

With the new architecture, the client embraced the AWS cloud, established multiregion cloud landing zones, and developed and productized a solution for employees and customers in less than four months.

Business benefits

• Established a cloud-native digital transformation foundation for the company using innovative managed serverless technologies
• Quickly introduced new rules based on business demands through a modular business rules engine
• Enabled 24/7/365 global availability by geographical compliance needs, including GDPR, with automatic backup

Technical benefits

• Enabled multiple parallel data ingestion capabilities and ingress protocols with high availability
• Successfully deployed a custom database for onboarding and authenticating customers through Okta to serve applications
• Consolidated and streamlined content ingestion tools, processes and operations
• Set up multilayered encryption mechanism for high-security customers
• Integrated an SSO–enabled SFTP mechanism with the existing customer Okta database
• Provided the ability to design and integrate high-performance APIs for ingestion and consumption
• Improved geospatial support through PostgreSQL 11 and PostGIS