How to use the Unified File Storage API
December 25, 2023
The Unified File Storage API allows your application to fetch your customer's files and folders from their favourite cloud storage provider, much like a file manager. These cloud storage providers include Google Drive, Microsoft OneDrive, Dropbox, and more - view all of them here.
This guide will walk you through the File Storage API and how to perform common tasks with it.
Understanding the StorageFile object
The object that is returned from the File Storage API can represent either a file or a folder. This is specified under the type field. Folders do not have a size nor a download_url.
type StorageFile = {
id?: string;
created_at?: string; // ISO date
updated_at?: string; // ISO date
name?: string;
description?: string;
parent_id?: string;
user_id?: string;
size?: number;
type?: 'FILE' | 'FOLDER';
mime_type?: string;
permissions?: {
user_id?: string;
group_id?: string;
roles?: any;
}[];
download_url?: string;
}
API reference: StorageFile data model
Get a list of files and folders
Most storage providers use a hierarchical file system to represent their contents as a tree. Imagine each node in the tree as either a file or a folder - if it's a folder, that node will have additional branches leading to other files and folders.
When calling the File Storage API List endpoint, we will return the contents (i.e. children) of a node. For example, calling:
GET /storage/{connection_id}/file
Will return the contents under the root node. Notice that no parameters are passed in this API call.
Include parent_id when calling the List endpoint to retrieve the contents of the node (i.e. folder) matching that ID, for example:
GET /storage/{connection_id}/file&parent_id=abc123
Will return the contents of the folder with ID abc123.
API reference: List files and folders
Traverse the file system to get all files
To traverse the file system and retrieve the contents of every subfolder, create a function to recursively call the List endpoint and pass in parent_id for each folder that is found. In other words, use tree recursion to retrieve all the contents of a storage provider's files:
import { UnifiedTo } from '@unified-api/typescript-sdk';
const sdk = new UnifiedTo({
security: {
jwt: '<YOUR_API_KEY_HERE>',
},
});
async function getAllFiles(connectionId: string, parentId?: string): Promise<StorageFile[]> {
const result = await sdk.storage.listStorageFiles({
connectionId,
parentId,
});
if (!result.storageFiles) {
return [];
}
let allFiles: StorageFile[] = [];
for (const item of result.storageFiles) {
if (item.type === 'FOLDER') {
// Recursively get files from subfolders
const subfolderFiles = await getAllFiles(connectionId, item.id);
allFiles = allFiles.concat(subfolderFiles);
} else {
allFiles.push(item);
}
}
return allFiles;
}
Retrieve the contents of a file
If you want to get the data contents of a file, either use the getFile API endpoint or take a File object from the listFiles API endpoint. Use the download_url field to fetch the contents of that file. How you fetch that data will be dependent on your programming language.
Important: The download_url expires after 1 hour. When you need to retrieve the contents of a file, call the getFile endpoint again to get a new download_url.
For example, to retrieve the contents of a file:
import { UnifiedTo } from '@unified-api/typescript-sdk';
const sdk = new UnifiedTo({
security: {
jwt: '<YOUR_API_KEY_HERE>',
},
});
export async function getFileContents(connectionId: string, id: string) {
// get file details
const result = await sdk.storage.getStorageFile({
id,
connectionId,
});
if (!result.storageFile) {
throw new Error('file not found');
}
if (!result.storageFile.downloadUrl) {
throw new Error('not a downloadable file');
}
// Get the data contents of a file
const contents = fetch(result.storageFile.downloadUrl, { Accept: result.storageFile.mimeType });
return contents;
}