NetApp volume: unknown state (missing volume on GUI)

I ran into an issue recently where the NetApp volume was not visible on the GUI. The system manager reported that aggregates were offline. However, that wasnt really the case. The volume was visible on the command line and an attempt to offline the volume produced the following error :

Modification not permitted. Unable to determine the state of the volume.
  

Issue: This was an issue in synchronizing data between WAFL and VLDB . 

Fix:  Run the 'debug vreport show' command in diagnostic mode. and then run 'debug vreport fix' to fix the error 

Here is the command output

CLUST::*> debug vreport show
aggregate Differences:

Name             Reason   Attributes
--------         -------  ---------------------------------------------------
<Volume Name>
                 Present both in VLDB and WAFL with differences
                          Node Name: Node1
                          Aggregate UUID: AggrgID
                          Aggregate State: online
                          Aggregate Raid Status: mixed_raid_type, hybrid
                          Aggregate HA Policy: sfo
                          Is Aggregate Root: false
  Differing Attribute: Volume Count (Try to fix volume differences using vreport for volume, and check again.)
    WAFL Value: 18
    VLDB Value: 19

volume Differences:

Name             Reason   Attributes
--------         -------  ---------------------------------------------------
VserverName:Volume Name
                 Present in VLDB Only
                          Node Name: Node1
                          Volume DSID:xxx MSID:yyyy
                          UUID: unknown
                          Aggregate Name: AggrName
                          Aggregate UUID: xxxx
                          Vserver UUID: xxxx
                          AccessType: READ_WRITE
                          StorageType: CLONE
                          Constituent Role: none

2 entries were displayed.
 

CLUST::*> debug vreport fix -type volume -object SFM:Volume_CVclone

Booting OnCommand virtual appliance

This is a classic!

You get the following when you boot the virtual appliance used for OCUnified Manager or OCPerformance Manager.


 


 Press ctrl + Alt + F2, and you will see more detailed output. Those details should help you fix the issue.




In this case, the file system is checked. Which is good thing.

wafl.vol.outOfInodes:notice]: file system on Volume xxx is out of inodes

If you receive this error on a NetApp volume, then it is an indication that the volume has a large number of files and that the volume has run out of inodes.

Here is how NetApp defines inodes.
The number of files a volume can contain is determined by how many inodes it has. An inode is a data structure that contains information about files. Volumes have both private and public inodes. Public inodes are used for files that are visible to the user; private inodes are used for files that are used internally by Data ONTAP. You can change only the maximum number of public inodes for a volume. You cannot affect the number of private inodes. 

Data ONTAP automatically sets the maximum number of public inodes for a newly created volume based on the size of the volume: 1 inode per 32 KB of volume size. When the size of a volume is increased, either directly by an administrator or automatically by Data ONTAP through the autosize feature, Data ONTAP also increases (if necessary) the maximum number of public inodes so there is at least 1 inode per 32 KB of volume size, until the volume reaches approximately 1 TB in size. Growing the volume greater than 1 TB in size does not automatically result in more inodes, because Data ONTAP does not automatically create more than 33,554,409 inodes


You can use the command 'maxfiles' to find the inode consumption status.
 filername> maxfiles cifs_vol1
Volume cifs_vol1: maximum number of files is currently 31876690 (31876690 used).


In order to resolve the issue, you can run the command 'maxfiles <volume name> <value>'

For ex:
filername> maxfiles cifs_vol1 40000000

The new maximum number of files will be rounded to 39999990.


 filername> maxfiles cifs_vol1
Volume cifs_vol1: maximum number of files is currently 39999990 (31876690 used).
 

Space reservation in NetApp

Before we discuss space reservation in NetApp, here is a look at the logical representation of aggregate, volume and LUN.





The LUN is created within a volume. There can be more than one LUN within a volume.


















When creating a LUN, you can reserve all the space required for the LUN or let the LUN consume capacity on need basis.

Scenario 1: Create 100GB LUN on 500GB volume with space reserved enabled. This immediately consumes 100GB capacity on the volume.

Scenario 2: Create 100GB LUN on 500GB volume with space reserved disabled. This creates a LUN of 100GB but doesn't consume the capacity on the volume. As data gets written to the LUN, the capacity is consumed.


NOTES:

1) LUN reservation is a property of the LUN and it is persistent across reboots, TO and GB.

2) There is a property on the volume called 'space gaurantee' or 'gaurantee'. When you create LUN with space reservation enabled then you should check the volume gaurantee settings as well. If volume gaurantee is none, then LUN space reservation might not actually work.

3)  LUN space allocation  is a feature that ensure the LUN follows the T10 SCSI standard. This standard requires that LUN remain online even when the LUN is full. If space allocation is disabled, then when the LUN is full (and it cant grow) it goes offline. However, if the space allocation option is enabled, then when the LUN if full (and it cant grow) it remain online. Hosts can read from the LUN but it cant write to the LUN.

4) LUN space allocation also enables storage to reclaim space automatically when the host deletes data.
  https://library.netapp.com/ecmdocs/ECMP1368845/html/GUID-A926B796-9C0D-4935-A84A-D9BF291487A4.html
https://library.netapp.com/ecmdocs/ECMP1368845/html/GUID-93D78975-6911-4EF5-BA4E-80E64B922D09.html