Point Cloud Storage Planning for GIS and LiDAR Data

Laser scanning, aerial LiDAR, and photogrammetry have changed how engineering firms capture existing conditions. They have also changed how much storage a firm needs. Point cloud storage planning helps you keep these large datasets fast to work with, safe to recover, and affordable to keep.

Why point cloud storage is different

Point clouds and LiDAR datasets are large, and they grow quickly. A single scanning project can add more data than years of ordinary drawings. GIS work adds imagery, rasters, and geodatabases on top of that.

These files also behave differently from documents. They are often already compressed, so storage tricks like deduplication save little space. In addition, processing software reads huge amounts of data at once, so slow storage makes staff wait.

Use storage tiers

Not all data needs the fastest storage. Instead, place data on tiers that match how often people use it.

• Working tier: fast storage for active projects, often solid state drives in a server or high-performance network storage.
• Nearline tier: larger, slower storage for recent projects that may need occasional access.
• Archive tier: low-cost storage for closed projects you must keep but rarely open.

Moving data between tiers on a schedule keeps fast storage available for the work that needs it. It also slows the growth of your most expensive storage.

Match the network to the data

Fast storage does not help if the network is slow. Many offices still run standard gigabit connections to workstations, which can be too slow for opening large point clouds from a server.

So consider faster links between the storage and the workstations that process scan data. Some firms also let power users process locally on fast workstation drives, then copy results back to the server. Either approach works, as long as the final data lands in the project record.

Check the switches too. A fast server connection helps little if an older switch in the middle limits every transfer.

Weigh cloud storage carefully

Cloud storage is useful for archives, offsite backups, and sharing with partners. However, it is not always the best place to work on active point clouds. Uploading takes time, and many providers charge fees when you download data.

Before you move large datasets to the cloud, estimate how often you will pull them back. Also check upload capacity at each office. For many firms, a hybrid approach works best, with fast local storage for active work and cloud storage for archive and backup.

Plan backup around size

Large datasets strain backup systems. A full backup may take longer than the overnight window allows, and a restore may take days. Therefore, plan backup with the data size in mind.

Separate raw scan data from derived products when you can. Raw data may only need one protected copy once someone verifies it, while active processing folders need frequent backups. Then test a restore of a large dataset so you know the real recovery time.

Also keep at least one backup copy offsite. A fire or flood at the office should not take out both your working data and every copy of it.

Set retention rules

Scan projects create many intermediate files, such as registration data, tiles, and test exports. Keeping all of them forever fills storage fast. On the other hand, deleting the wrong files can make a dataset impossible to rebuild.

Decide with your principals which files each project must keep after closeout. Typically that includes raw data, registered clouds, and final deliverables. Also check client contracts, because some specify how long you must retain project data.

Storage planning checklist

• Measure how much scan, LiDAR, and GIS data you added over the past year.
• Estimate growth based on upcoming projects and new equipment.
• Define working, nearline, and archive tiers with clear rules for moving data.
• Check network speed between storage and processing workstations.
• Estimate cloud upload times and download fees before committing.
• Set retention rules for raw, processed, and delivered data.
• Test a full restore of your largest active dataset.

How WEBIT helps

WEBIT helps engineering firms design point cloud storage that balances speed, safety, and cost. We assess current capacity and growth, recommend storage and network upgrades, and build tiered storage with image-based server backup and offsite copies. Our vCIO advisory and technology roadmaps also help you budget for storage before it runs out.

Explore our strategic IT services and cloud infrastructure services.

Key takeaways

• Point cloud storage needs capacity, speed, and a backup plan built for size.
• Use tiers so fast storage stays available for active work.
• Upgrade the network if staff open large scans from a server.
• Estimate cloud transfer times and fees before moving data.
• Test restores of your largest datasets.