In Solaris 11.1 Oracle decided to change the way that they stored filesystem share information. I personally loved the change, but that was without understanding what the change really was. For more information on how to enable NFS exports see my previous article here.
Setup Our Environment
Everything we are doing will be on some test file systems, with no data. So lets put everything in place.
# zfs create tank/fs1 # zfs set share=name=tank_fs1,path=/tank/fs1,prot=nfs,sec=sys,rw=@10.0.0.15 tank/fs1 name=tank_fs1,path=/tank/fs1,prot=nfs,sec=sys,rw=@10.0.0.15
Now lets create our child filesystem, and export it the same way.
# zfs create tank/fs1/sub1 # zfs set share=name=tank_fs1_sub1,path=/tank/fs1/sub1,prot=nfs,sec=sys,rw=@10.0.0.15 tank/fs1/sub1 name=tank_fs1_sub1,path=/tank/fs1/sub1,prot=nfs,sec=sys,rw=@10.0.0.15
Now this will simulate a situation that I was in. I had originally exported everything at a higher level (fs1) and then as the dataset grew it became necessary to export it at the child level (sub1) in order to have more granular control. So now we have two exports, and only need sub1.
Disable Unused NFS Export
Since fs1 is no longer needed lets disable its export.
# zfs set share.nfs=off tank/fs1
Lets take a look at fs1 to see if our configuration is still active.
# zfs get share tank/fs1
OK it is empty. But since I don’t need this any more I want to make sure that there are no remaining artifacts. So lets re-enable it and confirm that the configuration is gone.
# zfs set share.nfs=on tank/fs1 # zfs get share tank/fs1 NAME PROPERTY VALUE SOURCE tank/fs1 share name=tank_fs1,path=/tank/fs1,prot=nfs,sec=sys,rw=@10.0.0.15 local
OK, so our configuration is still there, but it is disabled by share.nfs=off. It must be kept elsewhere in the file system.
FYI, this is included in the ZFS metadata, so if you were to export you zpool and take it to another system, it would come over to the other system.
Explore ZFS Properties
Lets disable fs1 and start poking around.
# zfs set share.nfs=off tank/fs1
First thing to do is a get all. This will list all of the properties of the filesystem. I trimmed it back to only display the one that I found interesting.
# zfs get all tank/fs1 tank/fs1 share.* ... local
OK so lets look a little bit closer at this share.* property.
# zfs get share.* tank/fs1 bad property list: invalid property specifier 'share.*', use 'share.all' For more info, run: zfs help get
So share.* is not a valid property, it means that there are sub-properties, apparently share.all will display the sub-properties.
# zfs get share.all tank/fs1 NAME PROPERTY VALUE SOURCE tank/fs1 share.auto off local tank/fs1 share.autoname default tank/fs1 share.desc default tank/fs1 share.nfs off local tank/fs1 share.nfs.* ... default tank/fs1 share.point /tank/fs1 default tank/fs1 share.protocols - tank/fs1 share.smb off default tank/fs1 share.smb.* ... default
Oh boy, now we are getting somewhere, here is our share.nfs, and we can also see that share.nfs has sub-properties as well. Also notice that smb is also stored here in the same fashion.
# zfs get share.nfs.all tank/fs1 NAME PROPERTY VALUE SOURCE tank/fs1 share.nfs.aclok off default tank/fs1 share.nfs.anon default tank/fs1 share.nfs.charset.* ... default tank/fs1 share.nfs.cksum default tank/fs1 share.nfs.index default tank/fs1 share.nfs.log default tank/fs1 share.nfs.noaclfab off default tank/fs1 share.nfs.nosub off default tank/fs1 share.nfs.nosuid off default tank/fs1 share.nfs.sec default tank/fs1 share.nfs.sec.* ... default
Under share.nfs.all we have a lot of NFS parameters, the most important at this point is share.nfs.sec.all, so lets look there.
# zfs get share.nfs.sec.all tank/fs1 NAME PROPERTY VALUE SOURCE tank/fs1 share.nfs.sec.default.* ... default tank/fs1 share.nfs.sec.dh.* ... default tank/fs1 share.nfs.sec.krb5.* ... default tank/fs1 share.nfs.sec.krb5i.* ... default tank/fs1 share.nfs.sec.krb5p.* ... default tank/fs1 share.nfs.sec.none.* ... default tank/fs1 share.nfs.sec.sys.* ... default
We are getting warmer. Now we have the share.nfs.sec.sys.all to take a look at.
# zfs get share.nfs.sec.sys.all tank/fs1 NAME PROPERTY VALUE SOURCE tank/fs1 share.nfs.sec.sys.none default tank/fs1 share.nfs.sec.sys.ro default tank/fs1 share.nfs.sec.sys.root default tank/fs1 share.nfs.sec.sys.root_mapping default tank/fs1 share.nfs.sec.sys.rw default
OK well we have run out of levels to go deeper, but I still don’t see anything. Perhaps if we call the property by name?
# zfs get share.nfs.sec.sys.rw tank/fs1 NAME PROPERTY VALUE SOURCE tank/fs1 share.nfs.sec.sys.rw default
So that is empty as well. Perhaps that has something to do with the export being disabled? Lets confirm our export is disabled.
# zfs get share.nfs tank/fs1 NAME PROPERTY VALUE SOURCE tank/fs1 share.nfs off local
Doesn’t get any more disabled than that. Well lets check the sub1 file system.
# zfs get share.nfs tank/fs1/sub1 NAME PROPERTY VALUE SOURCE tank/fs1/sub1 share.nfs on local
So we can see that it is enabled, lets check the share property to see what our whole string is.
# zfs get share tank/fs1/sub1 NAME PROPERTY VALUE SOURCE tank/fs1/sub1 share name=tank_fs1_sub1,path=/tank/fs1/sub1,prot=nfs,sec=sys,rw=@10.0.0.15 local
Based on that string we should have share.nfs.sec.sys.rw = 10.0.0.15.
# zfs get share.nfs.sec.sys.rw tank/fs1/sub1 NAME PROPERTY VALUE SOURCE tank/fs1/sub1 share.nfs.sec.sys.rw default
OK well that was unexpected. Property is empty on the enabled export file system as well. We must be missing something.
Back to the Drawing Board
Out of pure desperation and a fair bit of luck. I was attempting to lookup the help command on zfs get, and accidentally typed list instead. But look at what I found.
# zfs help list usage: list [-rH][-d max] [-o property[,...]] [-t type[,...]] [-s property] ... [-S property] ... [filesystem|volume|snapshot|share] ...
There is a share file system type. Certainly that must have something to do with everything I am seeing?
# zfs list -t share NAME USED AVAIL REFER MOUNTPOINT tank/fs1%tank_fs1 - - - /tank/fs1 tank/fs1/sub1%tank_fs1_sub1 - - - /tank/fs1/sub1
Clearly we are onto something here. We have a share file system type for both our enabled and our disabled NFS export. Lets see what kind of properties these new file system types have.
ZFS File System Share Type
# zfs get all tank/fs1%tank_fs1 NAME PROPERTY VALUE SOURCE tank/fs1%tank_fs1 creation Sat Sep 21 10:09 2013 - tank/fs1%tank_fs1 mountpoint /tank/fs1 - tank/fs1%tank_fs1 share.* ... local tank/fs1%tank_fs1 zoned off default
Well there is a share.* so that must be what we are looking for. Lets try our previous command against this new file system and take a look at what we get.
# zfs get share.nfs.sec.sys.all tank/fs1%tank_fs1 NAME PROPERTY VALUE SOURCE tank/fs1%tank_fs1 share.nfs.sec.sys.none default tank/fs1%tank_fs1 share.nfs.sec.sys.ro default tank/fs1%tank_fs1 share.nfs.sec.sys.root default tank/fs1%tank_fs1 share.nfs.sec.sys.root_mapping default tank/fs1%tank_fs1 share.nfs.sec.sys.rw @10.0.0.15 local
Well there is that elusive read write statement.
Updating NFS Security Statements
Now lets see what happens when we change it.
# zfs set share.nfs.sec.sys.rw=@10.0.0.15:@10.0.0.16 tank/fs1%tank_fs1
Here we have added a second IP address to be allowed to read and write, on our disabled NFS export.
# zfs get share.nfs.sec.sys.all tank/fs1%tank_fs1 NAME PROPERTY VALUE SOURCE tank/fs1%tank_fs1 share.nfs.sec.sys.none default tank/fs1%tank_fs1 share.nfs.sec.sys.ro default tank/fs1%tank_fs1 share.nfs.sec.sys.root default tank/fs1%tank_fs1 share.nfs.sec.sys.root_mapping default tank/fs1%tank_fs1 share.nfs.sec.sys.rw @10.0.0.15:@10.0.0.16 local
The property is successfully updated on the share file system type. Lets see if it also enabled the NFS export.
# zfs get share tank/fs1
Still disabled, that is perfect. Now lets see what happens on an enabled NFS export.
# zfs set share.nfs.sec.sys.ro=@10.0.0.15:@10.0.0.16 tank/fs1/sub1%tank_fs1_sub1
Now we have added 2 new IPs to have read only access to this export. Lets check its effect.
# zfs get share.nfs.sec.sys.all tank/fs1/sub1%tank_fs1_sub1 NAME PROPERTY VALUE SOURCE tank/fs1/sub1%tank_fs1_sub1 share.nfs.sec.sys.none default tank/fs1/sub1%tank_fs1_sub1 share.nfs.sec.sys.ro @10.0.0.15:@10.0.0.16 local tank/fs1/sub1%tank_fs1_sub1 share.nfs.sec.sys.root default tank/fs1/sub1%tank_fs1_sub1 share.nfs.sec.sys.root_mapping default tank/fs1/sub1%tank_fs1_sub1 share.nfs.sec.sys.rw @10.0.0.15 local
The properties were successfully updated. Lets check the file system to see if the share properties were brought over.
# zfs get share tank/fs1/sub1 NAME PROPERTY VALUE SOURCE tank/fs1/sub1 share name=tank_fs1_sub1,path=/tank/fs1/sub1,prot=nfs,sec=sys,ro=@10.0.0.15:@10.0.0.16,rw=@10.0.0.15 local
That looks perfect, and lets ask NFS what it knows.
# showmount -e export list for server1: /tank/fs1/sub1 @10.0.0.15,@10.0.0.16,@10.0.0.15
Wonderful! That is exactly what we wanted. Now one final thing.
Removing NFS Export Configuration Permanently
Now since there is this whole different type of file system to store the share configuration, removing it becomes incredibly easy.
# zfs destroy tank/fs1%tank_fs1
That pretty much wraps up everything.