Ran into a situation on a customer’s CentOS server the other day where a service wasn’t working. Symptoms and error messages indicated the disk was full. However ‘$ df -h’ was showing ample free space. What the heck? Turned out the maximum number of files on the disk had been consumed. Technically speaking, the limiting factor was the number of inodes allocated to the volume. An inode is taken up for each file, directory and link on the file system. Inodes act like a database for the files on a file system and contain pointers to the actual information.
When a partition is created the maximum number of inodes is established, rather set in stone. There is no way to re-partition the number of inodes on the fly. In this particular case the volume was 75GB with 23GB free, but only 1,000,000 inodes were allocated to it. The temporary solution was to remove old files that were not needed to get the total number of files on the partition safely back below 1M. As soon as that was taken care of the system started working again.
Unix/Linux (and Mac of course) have the inode concept built into their file systems. To check out the inode status run ‘$ df -i’ to make sure you are not at risk of running out of those precious inodes.
user@host.com [~]# df -i Filesystem Inodes IUsed IFree IUse% Mounted on /dev/sda 49152000 8771724 40380276 18% /
inode related commands:
‘$ ls -i’ it will output the inode ids for each file / directory.
user@host.com [~]# ls -i1 1725516 access-logs@ 1721190 backups/ 1720340 dead.letter 1720652 etc/ 2173459 logs/ 1720654 mail/ 1720648 public_html/ 41845314 python@ 1729306 ssl/ 1720653 tmp/ 1720660 www@
The stat command will tell more details about the particular file / inode.
user@host.com [~]# stat public_html File: `public_html' Size: 4096 Blocks: 8 IO Block: 4096 directory Device: 800h/2048d Inode: 1720648 Links: 13 Access: (0750/drwxr-x---) Uid: ( 1058/user) Gid: ( 99/ nobody) Access: 2011-12-04 16:29:56.000000000 -0500 Modify: 2014-04-20 03:19:04.000000000 -0400 Change: 2014-05-17 00:00:11.000000000 -0400
To get a count of the inodes per folder under the current directory:
user@host.com [~]# find . -type f -printf "%h\n" | cut -d/ -f-2 | sort | uniq -c | sort -rn
5789 ./public_html
557 ./mail
555 ./tmp
205 ./logs
75 ./.cpanel
43 ./etc
25 .
13 ./.sqmaildata
10 ./.fontconfig
6 ./.subversion
6 ./.gnupg
6 ./.fantasticodata
5 ./.htpasswds
3 ./backups
2 ./.emacs.d
1 ./.ssh
1 ./public_ftp
1 ./.cpan
This can take forever so you may want to direct the output to a file (assuming you can spare an inode):
user@host.com [~]# find . -type f -printf "%h\n" | cut -d/ -f-2 | sort | uniq -c | sort -rn > inode_count.txt
For more information:
http://www.linux.org/threads/intro-to-inodes.4130/