Reliable and secure data storage is essential for Earth observation missions. EO satellite constellations generate vast amounts of data. Providing high-speed, easy access to these products is not a trivial exercise.
We have developed a high-performance, high-volume data management and access system with our FarEarth for SmallSats solution. Our processors create standard products in well-established formats. FarEarth Archives use existing technologies and expose APIs to ensure that you can easily integrate with your choice of storage provider and existing systems.
Data storage architectures
We seamlessly integrate with the most popular data storage architectures. Whether you are using Microsoft’s Azure Blob Storage, Amazon’s S3 buckets, an FTP server, or Google Drive, our Archive can reliably organise the data for your applications.
Our offerings include significant storage, typically a year’s worth of processed data. Storage capacity can be easily expanded as you grow. We can also securely integrate your existing storage accounts.
Data management
Keeping data organised can be daunting. We understand that you may prefer to organise your data in a specific way. In FarEarth, you can choose to sort the data into a folder structure using properties from the products themselves. Below is an example of such a configuration.
You may not want to keep all products indefinitely. For example, you might want to expose Level 2 products from the last three months to users, while older raw products are stored with a cheaper storage provider. FarEarth’s Archive supports rule-based data management. The rolling Archive example shown below is configured to automatically remove the oldest products as the storage fills up.
Data access
An effective data archiving solution should conceal the complexities of the underlying storage from data users. Users should not be concerned whether a particular product is stored on a high-availability medium or a slower cold storage account. Finding the data and getting a URL to the files should be seamless and independent of underlying storage technologies.
FarEarth subscriptions include access to a Catalogue, making it easy to find and inspect products in the Archive. We support OpenAPI access with STAC-compatible products. We have expanded the “Collections” concept of STAC catalogues to dynamically update as new products are added. It removes the need to update existing products when a new collection is defined. A configuration example is shown below, filtering all Level 1C products.
All of this means nothing if access to the data isn’t tightly regulated. We have designed FarEarth data interfaces to work on the principle that a user must have a valid, short-lived, access-constrained, and pre-signed URL to retrieve data. The full access credentials never leave the local system. Each data access request is authorised by FarEarth and backed by industry-standard authentication from Microsoft’s Entra (B2C) services. APIs maximise file transfer performance by providing URLs to individual files or a zipped copy of an entire product.
Enterprise-grade features
A fully automated backup system ensures that you never lose your data.
Suppose you need to move data to another location. FarEarth has a migration component that can safely move all or part of the Archive while maintaining the integrity of the Catalogue.
The Archive can synchronise between files stored in the Archive and the Catalogue. This is useful if you have removed data from the storage location. This enables bulk data to be imported into an existing Archive from a copy of another Archive.
The Archive can mirror other data sources and make them transparently available to FarEarth users. This is useful for giving FarEarth processors access to third-party ancillary files. No need to integrate these sources manually; you simply request the relevant files from our Archive, and FarEarth will fetch them on your behalf. FarEarth will authorise the transfer and manage the third-party credentials transparently.
The Archive supports the FarEarth processing systems by providing efficient access to storage, while giving you complete control over the format, structure, and access to the data. All data in FarEarth remains the property of the satellite owner, but we also make it easy for you to share and disseminate your data.