I’ve found XFS more well suited to extremely large filesystems with possibly many large files. I’ve had a functioning 3.6TB XFS filesystem for over 2 years now with no problems. Definately works better than ext3, etc at that size (especially when dealing with many large files and lots of I/O). What you get with ZFS … Read more
One aspect of my job is designing and building large-scale storage systems (often known as “SANs”, or “Storage Area Networks”). Typically, we use a tiered approach with SSD’s and HDD’s combined. That said, each one has specific benefits. SSD’s almost always have a higher Cost-per-Byte. I can get 10k SAS 4kn HDD’s with a cost-per-gigabyte … Read more
First it is best to define the difference between a block device and filesystem. This is easier grasped if you are familiar with UNIX because it makes an objective distinction between the two things. Still the same applies to Windows. A block device is a handle to the raw disk. Such as /dev/sda for a … Read more
I’d recommend using a regular file system instead of databases. Using file system is easier than a database, you can use normal tools to access files, file systems are designed for this kind of usage etc. NTFS should work just fine as a storage system. Do not store the actual path to database. Better to … Read more
Server hard-drive capacities are miniscule compared to desktop hard-drive capacities. 450 and 600GB are not uncommon sizes to see in brand new servers, and you could buy many 4TB SATA desktop drives for the price of one 600GB SAS (server) hard drive. Your SATA hard-drive in your desktop PC at home is like a muscle … Read more
This is going to be a function of your workload and the class of drive you purchase… In my server deployments, I have not had a properly-spec’d SSD fail. That’s across many different types of drives, applications and workloads. Remember, not all SSDs are the same!! So what does “properly-spec’d” mean? If your question is … Read more
RAID: Why and When RAID stands for Redundant Array of Independent Disks (some are taught “Inexpensive” to indicate that they are “normal” disks; historically there were internally redundant disks which were very expensive; since those are no longer available the acronym has adapted). At the most general level, a RAID is a group of disks … Read more
I’m in charge of some applications that manage many TB of images. We’ve found that storing file paths in the database to be best. There are a couple of issues: database storage is usually more expensive than file system storage you can super-accelerate file system access with standard off the shelf products for example, many web servers … Read more
From GitHub’s documentation: GitHub doesn’t have any set disk quotas. We try to provide abundant storage for all Git repositories, within reason. Keeping repositories small ensures that our servers are fast and downloads are quick for our users. and In addition, we place a strict limit of files exceeding 100 MB in size. Now for the … Read more