With the base software now installed, it's time to test it.
First off, we need to check if the web server is started. Is isn't:
root@ds1 bin]#/usr/sbin/smcwebserver status
Sun Java(TM) Web Console is stopped
root@ds1 bin]#/usr/sbin/smcwebserver start
Starting Sun Java(TM) Web Console Version 3.0.3 ...
The console is running
I was then able to login into the web console at https://ds1:6789 using my standard local user account. To initialise the Directory Service Control Center apparently requires root access, so against my better judgement I logged into the web interface as root.
The Control Center did it's thing, and to my delight it even recommended that I return back to using a non-privileged user again now that the initialisation was complete.
Initial indications were that everything seemed mightily slow, but it may speed up with time.
The authentication to the webconsole can be done as any local user and once you select the DSCC, you're asked for admin authentication to the DS itself. Makes sense, so basically the Sun directory server is just piggybacking on the existing web management console for remote administration. There doesn't appear to be a local ldap/management client.
It quickly became obvious that the web interface didn't have any directory servers registered. It seems that I setup what was needed for the DS server itself, but not an actual ldap directory.
I tried created a new directory, but failed with Could not contact the DSCC agent on ds1. Use the command cacaoadm to check that the DSCC agent is installed and running on port 11162.
So:
root@ds1 bin]#cacaoadm status
Cannot find property: [cacao.embedded].
Doh!
Google provided http://bugs.opensolaris.org/view_bug.do;jsessionid=4694b06edf8cd25d148e6a54c8fa?bug_id=6745235
which shows this as being a bug in snv_97. This host is using snv_95, but I seemed to be having the same issue:
root@ds1 bin]#pkginfo -l SUNWcacaort | grep VERSION
VERSION: 2.0,REV=15
and yet the snv_95 DVD came with:
root@angelous Product]#cat /mnt/tmp/Solaris_11/Product/SUNWcacaort/pkginfo | grep VERS
VERSION=2.2.0.1,REV=2008.06.06
So I replaced SUNWcacaort, SUNWcacaosvr and SUNWcacaodtrace. There were errors along the way, but at least the command didn't return errors now.
The service isn't started by default:
root@ds1 ~]#cacaoadm enable
root@ds1 ~]#cacaoadm status
default instance is ENABLED at system startup.
default instance is not running.
root@ds1 ~]#cacaoadm start
root@ds1 ~]#cacaoadm status
default instance is ENABLED at system startup.
Smf monitoring process:
12649
12650
Uptime: 0 day(s), 0:1
Lots more RAM was being eaten up now, but moving on...
Cacoa was only listening on localhost, so to get dsee working with it, I had to rerun the registration from dsccsetup
root@ds1 ~]#cacaoadm list-params | grep network-bind
network-bind-address=127.0.0.1
root@ds1 ~]#/opt/SUNWdsee/dscc6/bin/dsccsetup status
***
DSCC Application is registered in Sun Java (TM) Web Console
***
DSCC Agent is not registered in Cacao
***
DSCC Registry has been created
Path of DSCC registry is /var/opt/SUNWdsee/dscc6/dcc/ads
Port of DSCC registry is 3998
***
root@ds1 ~]#/opt/SUNWdsee/dscc6/bin/dsccsetup cacoa-reg
Invalid subcommand cacoa-reg
For more information, see dsccsetup --help.
root@ds1 ~]#/opt/SUNWdsee/dscc6/bin/dsccsetup cacao-reg
Registering DSCC Agent in Cacao...
Checking Cacao status...
Stopping Cacao...
Enabling remote connections in Cacao ...
Starting Cacao...
DSCC agent has been successfully registered in Cacao.
root@ds1 ~]#cacaoadm list-params | grep network-bind
network-bind-address=0.0.0.0
Better!
Creating the new ldap server failed the first time, for two reasons.
1. The user that I was using didn't have the right to create a service with a low port number.
Solved using RBAC:
root@ds1 ~]#usermod -K defaultpriv=net_privaddr ds
2. The directory that I specified for the new domain didn't exist yet, and the used didn't have permission to create it. I rather just expected this to work.
root@ds1 SUNWdsee]#mkdir griffous.net
root@ds1 SUNWdsee]#chown ds:ds griffous.net/
root@ds1 SUNWdsee]#cd griffous.net/
root@ds1 griffous.net]#pwd
/opt/SUNWdsee/griffous.net
However this too didn't quite work, because the directory already existed. Finally I granted ds access to /opt/SUNWdsee, just to get the directory created.
I didn't manage to get the server started on a priviledged port, but it did come up on a non-priviledged one. I'll troubleshoot that further tomorrow.
Sunday, October 26, 2008
Saturday, October 25, 2008
Centralised Authentication on Solaris part #2
Installing a whole root zones take time. A lot of it.
# time zoneadm -z ds1 install
A ZFS file system has been created for this zone.
Preparing to install zone.
Creating list of files to copy from the global zone.
Copying <210541> files to the zone.
Initializing zone product registry.
Determining zone package initialization order.
Preparing to initialize <1334> packages on the zone.
Initialized <1334> packages on zone.
Zone is initialized.
The file contains a log of the zone installation.
real 65m58.822s
user 2m38.165s
sys 8m11.322s
Given just how long it took, one starts to wonder at the advantages of using zones at all, over just using stand alone virtual machines. A full VM is much more portable, and I wonder just how much management you save by using a whole root zone.
A topic for another day...
With my newly initialised zone, I proceeded to run the installer again.
This post is going to be very text heavy, but it serves as some self documentation for me, and maybe an outline for anyone else that hits this.
Choose Software Components - Main Menu
-------------------------------
Note: "* *" indicates that the selection is disabled
[ ] 1. Web Server 7.0 Update 1
[ ] 2. Directory Preparation Tool 6.4
[ ] 3. Application Server Enterprise Edition 8.2 Patch 2
[ ] 4. Directory Server Enterprise Edition 6.2
[ ] 5. Monitoring Console 1.0 Update 1
[ ] 6. High Availability Session Store 4.4.3
[ ] 7. Access Manager 7.1
[ ] 8. Message Queue 3.7 UR2
* * Java DB 10.2.2.1
[ ] 10. All Shared Components
Enter a comma separated list of products to install, or press R to refresh
the list [] {"<" goes back, "!" exits}: 4
Enter a comma separated list of components to install (or A to install all )
[A] {"<" goes back, "!" exits}
*[X] 1. Directory Service Control Center
*[X] 2. Directory Server Command-Line Utility
Next came the share components that failed in the sparse zone:
Shared Component Upgrades Required
-----------------------------------
The shared components listed below are currently installed. They will be
upgraded for compatibility with the products you chose to install.
Component Package
--------------------
Cacao SUNWcacaort
2.2.0.1 (installed)
2.0:PATCHES:123896-03 (required)
Cacao SUNWcacaowsvr
2.2.0.1 (installed)
2.0:PATCHES:123897-03 (required)
JavaActivationFramework SUNWjaf
8.0.0.0 (installed)
8.1 (required)
JavaMail SUNWjmail
8.0.0.0 (installed)
8.1 (required)
SunWebConsole SUNWmctag
3.0.2 (installed)
3.0.2:PATCHES:125953-05 (required)
SunWebConsole SUNWmconr
3.0.2 (installed)
3.0.2:PATCHES:125951-05 (required)
SunWebConsole SUNWmcon
3.0.2 (installed)
3.0.2:PATCHES:125953-05 (required)
SunWebConsole SUNWmcos
3.0.2 (installed)
3.0.2:PATCHES:125951-05 (required)
Ant SUNWant
11.11.0 (installed)
11.12.0 (required)
Enter 1 to upgrade these shared components and 2 to cancel [1] {"<" goes
back, "!" exits}:1
I told it to upgrade everything it needed.
Installation Directories
------------------------
Enter the name of the target installation directory for each product:
Directory Server [/opt/SUNWdsee] {"<" goes back, "!" exits}:
Directory Preparation Tool [/opt/SUNWcomds] {"<" goes back, "!" exits}:
Checking System Status
Available disk space... : Checking .... OK
Memory installed... : Checking .... OK
Swap space installed... : Checking .... OK
Operating system patches... : Checking .... OK
Operating system resources... : Checking .... OK
System ready for installation
System Ready for Installation. Memory detection is disabled in a non-global zone.
Neat, it worked out that it's not real. On with the installation, nearly.
Screen for selecting Type of Configuration
1. Configure Now - Selectively override defaults or express through
2. Configure Later - Manually configure following installation
Select Type of Configuration [1] {"<" goes back, "!" exits} 1
I opted to configure everything up front, since I may not know how to change the defaults easily later
Specify Common Server Settings
Enter Host Name [ds1] {"<" goes back, "!" exits}
Enter DNS Domain Name [griffous.net] {"<" goes back, "!" exits}
Enter IP Address [192.168.1.74] {"<" goes back, "!" exits}
Enter Server admin User ID [admin] {"<" goes back, "!" exits}
Enter Admin User's Password (Password cannot be less than 8 characters) []
{"<" goes back, "!" exits}
Confirm Admin User's Password [] {"<" goes back, "!" exits}
Enter System User [ds] {"<" goes back, "!" exits} ds
Enter System Group [ds] {"<" goes back, "!" exits} ds
Directory Server: Create Directory Instance
Directory Server Console requires Directory Server, but does not require
a directory instance.
Although not a requirement, you can create a directory instance now
during installation.
Create a directory instance (in addition to installing Directory Server)?
1. Yes
2. No
Enter 1 or 2 [1] {"<" goes back, "!" exits} 1
I opted to setup the directory while at it, accepting all the defaults.
Directory Server: Specify Instance Creation Information
Enter Instance Directory [/var/opt/SUNWdsee/dsins1] {"<" goes back, "!"
exits}
Enter Instance Port [389] {"<" goes back, "!" exits}
Enter Instance SSL Port [636] {"<" goes back, "!" exits}
Directory Manager DN [cn=Directory Manager] {"<" goes back, "!" exits}
System User [root] {"<" goes back, "!" exits} ds
System Group [root] {"<" goes back, "!" exits} ds
Enter Instance password (At least 8 characters long) [] {"<" goes back, "!"
exits}
Retype Password [] {"<" goes back, "!" exits}
Enter Suffix [dc=griffous,dc=net] {"<" goes back, "!" exits}
Ready to Install
----------------
The following components will be installed.
Product: Java Enterprise System Identity Management Suite
Uninstall Location: /var/sadm/prod/SUNWident-entsys5u1
Space Required: 138.79 MB
---------------------------------------------------------
Sun Java(TM) System Directory Preparation Tool
Sun Java(TM) System Directory Server Enterprise Edition 6.2
Sun Java(TM) System Directory Server Enterprise Edition 6.2 Command-Line
Utilities
Java Enterprise System Directory Server 6.2 Core Server
Java Enterprise System Directory Service Control Center
1. Install
2. Start Over
3. Exit Installation
What would you like to do [1] {"<" goes back, "!" exits}?1
This failed, so I tried again, this time with everything as root.
Java Enterprise System Identity Management Suite
|-1%--------------25%-----------------50%-----------------75%--------------100%|
Installation Complete
# time zoneadm -z ds1 install
A ZFS file system has been created for this zone.
Preparing to install zone
Creating list of files to copy from the global zone.
Copying <210541> files to the zone.
Initializing zone product registry.
Determining zone package initialization order.
Preparing to initialize <1334> packages on the zone.
Initialized <1334> packages on zone.
Zone
The file contains a log of the zone installation.
real 65m58.822s
user 2m38.165s
sys 8m11.322s
Given just how long it took, one starts to wonder at the advantages of using zones at all, over just using stand alone virtual machines. A full VM is much more portable, and I wonder just how much management you save by using a whole root zone.
A topic for another day...
With my newly initialised zone, I proceeded to run the installer again.
This post is going to be very text heavy, but it serves as some self documentation for me, and maybe an outline for anyone else that hits this.
Choose Software Components - Main Menu
-------------------------------
Note: "* *" indicates that the selection is disabled
[ ] 1. Web Server 7.0 Update 1
[ ] 2. Directory Preparation Tool 6.4
[ ] 3. Application Server Enterprise Edition 8.2 Patch 2
[ ] 4. Directory Server Enterprise Edition 6.2
[ ] 5. Monitoring Console 1.0 Update 1
[ ] 6. High Availability Session Store 4.4.3
[ ] 7. Access Manager 7.1
[ ] 8. Message Queue 3.7 UR2
* * Java DB 10.2.2.1
[ ] 10. All Shared Components
Enter a comma separated list of products to install, or press R to refresh
the list [] {"<" goes back, "!" exits}: 4
Enter a comma separated list of components to install (or A to install all )
[A] {"<" goes back, "!" exits}
*[X] 1. Directory Service Control Center
*[X] 2. Directory Server Command-Line Utility
Next came the share components that failed in the sparse zone:
Shared Component Upgrades Required
-----------------------------------
The shared components listed below are currently installed. They will be
upgraded for compatibility with the products you chose to install.
Component Package
--------------------
Cacao SUNWcacaort
2.2.0.1 (installed)
2.0:PATCHES:123896-03 (required)
Cacao SUNWcacaowsvr
2.2.0.1 (installed)
2.0:PATCHES:123897-03 (required)
JavaActivationFramework SUNWjaf
8.0.0.0 (installed)
8.1 (required)
JavaMail SUNWjmail
8.0.0.0 (installed)
8.1 (required)
SunWebConsole SUNWmctag
3.0.2 (installed)
3.0.2:PATCHES:125953-05 (required)
SunWebConsole SUNWmconr
3.0.2 (installed)
3.0.2:PATCHES:125951-05 (required)
SunWebConsole SUNWmcon
3.0.2 (installed)
3.0.2:PATCHES:125953-05 (required)
SunWebConsole SUNWmcos
3.0.2 (installed)
3.0.2:PATCHES:125951-05 (required)
Ant SUNWant
11.11.0 (installed)
11.12.0 (required)
Enter 1 to upgrade these shared components and 2 to cancel [1] {"<" goes
back, "!" exits}:1
I told it to upgrade everything it needed.
Installation Directories
------------------------
Enter the name of the target installation directory for each product:
Directory Server [/opt/SUNWdsee] {"<" goes back, "!" exits}:
Directory Preparation Tool [/opt/SUNWcomds] {"<" goes back, "!" exits}:
Checking System Status
Available disk space... : Checking .... OK
Memory installed... : Checking .... OK
Swap space installed... : Checking .... OK
Operating system patches... : Checking .... OK
Operating system resources... : Checking .... OK
System ready for installation
System Ready for Installation. Memory detection is disabled in a non-global zone.
Neat, it worked out that it's not real. On with the installation, nearly.
Screen for selecting Type of Configuration
1. Configure Now - Selectively override defaults or express through
2. Configure Later - Manually configure following installation
Select Type of Configuration [1] {"<" goes back, "!" exits} 1
I opted to configure everything up front, since I may not know how to change the defaults easily later
Specify Common Server Settings
Enter Host Name [ds1] {"<" goes back, "!" exits}
Enter DNS Domain Name [griffous.net] {"<" goes back, "!" exits}
Enter IP Address [192.168.1.74] {"<" goes back, "!" exits}
Enter Server admin User ID [admin] {"<" goes back, "!" exits}
Enter Admin User's Password (Password cannot be less than 8 characters) []
{"<" goes back, "!" exits}
Confirm Admin User's Password [] {"<" goes back, "!" exits}
Enter System User [ds] {"<" goes back, "!" exits} ds
Enter System Group [ds] {"<" goes back, "!" exits} ds
Directory Server: Create Directory Instance
Directory Server Console requires Directory Server, but does not require
a directory instance.
Although not a requirement, you can create a directory instance now
during installation.
Create a directory instance (in addition to installing Directory Server)?
1. Yes
2. No
Enter 1 or 2 [1] {"<" goes back, "!" exits} 1
I opted to setup the directory while at it, accepting all the defaults.
Directory Server: Specify Instance Creation Information
Enter Instance Directory [/var/opt/SUNWdsee/dsins1] {"<" goes back, "!"
exits}
Enter Instance Port [389] {"<" goes back, "!" exits}
Enter Instance SSL Port [636] {"<" goes back, "!" exits}
Directory Manager DN [cn=Directory Manager] {"<" goes back, "!" exits}
System User [root] {"<" goes back, "!" exits} ds
System Group [root] {"<" goes back, "!" exits} ds
Enter Instance password (At least 8 characters long) [] {"<" goes back, "!"
exits}
Retype Password [] {"<" goes back, "!" exits}
Enter Suffix [dc=griffous,dc=net] {"<" goes back, "!" exits}
Ready to Install
----------------
The following components will be installed.
Product: Java Enterprise System Identity Management Suite
Uninstall Location: /var/sadm/prod/SUNWident-entsys5u1
Space Required: 138.79 MB
---------------------------------------------------------
Sun Java(TM) System Directory Preparation Tool
Sun Java(TM) System Directory Server Enterprise Edition 6.2
Sun Java(TM) System Directory Server Enterprise Edition 6.2 Command-Line
Utilities
Java Enterprise System Directory Server 6.2 Core Server
Java Enterprise System Directory Service Control Center
1. Install
2. Start Over
3. Exit Installation
What would you like to do [1] {"<" goes back, "!" exits}?1
This failed, so I tried again, this time with everything as root.
Java Enterprise System Identity Management Suite
|-1%--------------25%-----------------50%-----------------75%--------------100%|
Installation Complete
Centralised Authentication on Solaris
It's a big topic. Some might even call it a bit daunting.
Most importantly this is a subject that I didn't feel I knew enough about, and so began the journey of discovery to learn what one must do to have single sign on (SSO) under Solaris.
Microsoft's AD really does make it a bit easy for windows admins - there really is just the one choice, and like it or not you're going to be doing it the one way. About the only weird thing is that they persist with making the very standard task of setting up a domain controller, a command line app (dcpromo.exe), despite windows being very obviously a GUI centric world.
On the Solaris/Linux front we're too spoilt for choice. There are a good number of centralised directory/LDAP systems out there and I spent the best part of a day just trying to catch up on all the naming conventions and versions.
The major players appear to be:
Apache Directory Server
OpenDS
OpenLDAP
Sun Java System Directory Server
I had a look at the last three in some depth.
OpenDS is an entirely java based ldap server, which is a bit...odd, but more importantly to me, it doesn't appear to have any kind of a integrated gui front end whatsoever.
Now I know what you're thinking, it's Solaris, if you don't want command line, then pick up your toys and go play with Windows!
I've already deployed OpenLDAP on linux, and have run it for around 4 years powering my home network. It's been solid but it's really painful having to use command line tools for even the most mundane of updates. By the time you throw some kind of SSL into the mix, it becomes really ugly to manage and maintain. I'm sorry - this time around I wanted a GUI and that's that.
Which brings us to the Sun Java Directory Server - a product which goes by so many many names it's VERY hard to figure out what you are working on. Being a Sun product designed for Solaris it did seem the obvious tool for the job right from the start, but I wanted to at least give some of the other choices some review before making a start.
Having made the decision to go with the Sun DS, I created a zone on supernova and started the install.
Details in the next post.
Most importantly this is a subject that I didn't feel I knew enough about, and so began the journey of discovery to learn what one must do to have single sign on (SSO) under Solaris.
Microsoft's AD really does make it a bit easy for windows admins - there really is just the one choice, and like it or not you're going to be doing it the one way. About the only weird thing is that they persist with making the very standard task of setting up a domain controller, a command line app (dcpromo.exe), despite windows being very obviously a GUI centric world.
On the Solaris/Linux front we're too spoilt for choice. There are a good number of centralised directory/LDAP systems out there and I spent the best part of a day just trying to catch up on all the naming conventions and versions.
The major players appear to be:
Apache Directory Server
OpenDS
OpenLDAP
Sun Java System Directory Server
I had a look at the last three in some depth.
OpenDS is an entirely java based ldap server, which is a bit...odd, but more importantly to me, it doesn't appear to have any kind of a integrated gui front end whatsoever.
Now I know what you're thinking, it's Solaris, if you don't want command line, then pick up your toys and go play with Windows!
I've already deployed OpenLDAP on linux, and have run it for around 4 years powering my home network. It's been solid but it's really painful having to use command line tools for even the most mundane of updates. By the time you throw some kind of SSL into the mix, it becomes really ugly to manage and maintain. I'm sorry - this time around I wanted a GUI and that's that.
Which brings us to the Sun Java Directory Server - a product which goes by so many many names it's VERY hard to figure out what you are working on. Being a Sun product designed for Solaris it did seem the obvious tool for the job right from the start, but I wanted to at least give some of the other choices some review before making a start.
Having made the decision to go with the Sun DS, I created a zone on supernova and started the install.
Details in the next post.
Sunday, September 14, 2008
zonadm & zfs clones
Having spent nearly 2 hours chasing my tail on this now, it's time to blog about it for future safekeeping - that and google didn't find any good pages discussing this problem for me, so perhaps it will benefit someone else when google crawls this entry.
Firstly, my environment:
I'm using Solaris express community edition, build 95, on a machine named supernova.
Supernova has 2 zpools. rpool is the default zfs-root pool that we all know about, the second is an 8 disk raidz zpool, named (very creatively) Z
Background:
I was trying to clone a zone that I'd [mostly] setup earlier, so that my latest zone didn't waste a whole bunch of disk space with duplicate data.
Once/if ZFS de-duplication comes along, perhaps this will become a non-issue, but for now I really like the idea of creating new zones in under 1 second, and not wasting disk space doing it. As an added bonus, there is a chance performance will be slightly better, due to shared caching of data blocks,
I already have 2 zones setup on supernova: dns, and proxy, and tonight was time to make a start on a mail server zone to replace my suse-linux, xen based mail server.
Unfortunately it's now the end of the night, and I'm still no closer to actually setting up the new mail server, but I did at least succeed in getting a zfs cloned "mail" zone running and configured. And here it is, captured for all time
The problem was that zoneadm was simply refusing to actually clone my zone. Instead it was copying all the data across.
Here is what I started off with:
NAME USED AVAIL REFER MOUNTPOINT
Z 1.30T 567G 43.0G /Z
Z/backups 64.7G 567G 34.8K none
Z/backups/angelous 64.7G 567G 44.4G /Z/backups/angelous
Z/backups/angelous/home 20.3G 567G 1.77G /Z/backups/angelous/home
Z/backups/supernova 33.0K 567G 33.0K /Z/backups/supernova
Z/media 1.15T 567G 1.15T /Z/media
Z/storage 45.3G 567G 45.3G /Z/storage
Z/zones 1.79G 567G 36.5K none
Z/zones/dns 15.7M 567G 374M /zones/dns
Z/zones/proxy 1.70G 567G 1.69G /zones/proxy
rpool 6.66G 30.0G 36K /rpool
rpool/ROOT 5.65G 30.0G 18K legacy
rpool/ROOT/snv_95 5.65G 30.0G 5.36G /
rpool/export 66.5K 30.0G 28K /export
rpool/export/home 38.5K 30.0G 38.5K /export/home
We'll be focusing on the lines that I've bolded.
Firstly lets look at which directories map to which zfs's and for that matter, zpools.
rpool is / and most of Z is mounted under /Z/
zones however are under /zones/* which seemed logical enough to me.
I mainly use my raidz for storage of media and backups, whereas my zones as part of the core OS, are mounted directly off root as you would expect.
I wanted to make use of the 8-disk raidZ for both redundancy and performance for the zones, so putting them on the rpool was not the goal.
Z/zones was set with no mountpoint.
I did this because it doesn't actually contain any data, and since all zones will have their own zfs datasets, it will never need to contain any actual data. I've been caught out before when I've created a series of nested zfs's, and copied data into an "admin" parent directory and then "lost" it as zfs have mapped over the same namespace. Sure, in unix-speak everything just maps on top, but that's also a curse sometimes, and things can get hidden!
So, Z/zones has no mountpoint, but the children are mounted under it in /zones/dns and /zones/proxy.
Where exactly is /zones coming from, if not from Z/zones?
When I first started it was coming from / which is rpool, also a zfs fs, but the wrong zpool - which is important as I'll come to later.
I'd been running quite happily like this ever since upgrading supernova to snv_95, and dns is actually a clone of proxy - so I know that clones do work.
Yet tonight when I tried doing a zoneadm -z mail clone proxy, it was doing two things wrong. It wasn't creating a dedicated zfs for mail, and it wasn't cloning, it was copying... it even told me so.
Why? I'm running zfs root so I am sure that the prerequisite of the zonepath being on a zfs was being met!
The zonepath for mail, according to zonecfg -z mail was as follows:
zonename: mail
zonepath: /zones/mail
...which was exactly the same layout as dns and proxy.
The issue was all around the /zones directory, which technically didn't exist on my Z zpool at all, it was from /.
Now clearly zfs isn't going to be able to clone across zpools, so that explains why the clone failed. An warning/explanation would have been nice though!
But what about simply creating a zfs dataset on rpool then?
I think it wasn't doing that because zoneadm is being quite clever about it's use of naming and zfs mountpoint inheritance.
Based on my mucking around on this, it seems that zoneadm looks at the path that you've given it, and takes the parent dataset from this, and then attempts to create a child based on the path that you've provided.
I set the zonepath to /zones/mail, and I think the code then tries to access a /zones zfs. Now in my case, /zones wasn't a zfs... it was in a zfs, but it wasn't itself a zfs.
I guess zoneadm likes for zones to be directly below dedicated zfs datasets, as it can then inherit everything that it needs. One has to remember that while zfs datasets have a concept of heirachy, that doesn't actually necessarily match the logic directory layout on the disk.
In fact I do just this already with Z
The root of the Z zfs is under /Z, however /Z/zones/dns is up a directory, and then down another, which just happens to also be a zfs dataset, but until my upgrade to snv_95, it was ufs instead.
zfs and the unix mount anywhere system is almost too flexible sometimes, it gets confusing!
For zoneadm checked the provided path for a parent zfs, but because /zones wasn't a zfs, it had no way of knowing that I meant for it to create a child of Z/zones, since it wasn't mounted there.
So in the end, I was having a double failure, which results in my very weird results. At least I know understand why it was happening.
The fix was easy: I shutdown my zones, zfs umounted them, and then deleted the /zones directory from / (rpool)
With that out of the way, I set the mountpoint of Z/zones to be /zones, issued a zfs mount -a, and repeated my zonedam -z mail clone proxy.
This time around we were away, and fastforwarding a bit; here is where I'm at now:
NAME USED AVAIL REFER MOUNTPOINT
Z 1.30T 567G 43.0G /Z
Z/backups 64.7G 567G 34.8K none
Z/backups/angelous 64.7G 567G 44.4G /Z/backups/angelous
Z/backups/angelous/home 20.3G 567G 1.77G /Z/backups/angelous/home
Z/backups/supernova 33.0K 567G 33.0K /Z/backups/supernova
Z/media 1.15T 567G 1.15T /Z/media
Z/storage 45.3G 567G 45.3G /Z/storage
Z/zones 1.79G 567G 36.5K /zones
Z/zones/dns 15.7M 567G 374M /zones/dns
Z/zones/mail 83.1M 567G 1.72G /zones/mail
Z/zones/proxy 1.70G 567G 1.69G /zones/proxy
rpool 6.66G 30.0G 36K /rpool
rpool/ROOT 5.65G 30.0G 18K legacy
rpool/ROOT/snv_95 5.65G 30.0G 5.36G /
rpool/export 67.5K 30.0G 28K /export
rpool/export/home 39.5K 30.0G 39.5K /export/home
Lesson to be learned, if you wish to have zfs clone based zones, the parent directory of zonepath MUST be a zfs dataset, or zoneadm will get very confused.
Firstly, my environment:
I'm using Solaris express community edition, build 95, on a machine named supernova.
Supernova has 2 zpools. rpool is the default zfs-root pool that we all know about, the second is an 8 disk raidz zpool, named (very creatively) Z
Background:
I was trying to clone a zone that I'd [mostly] setup earlier, so that my latest zone didn't waste a whole bunch of disk space with duplicate data.
Once/if ZFS de-duplication comes along, perhaps this will become a non-issue, but for now I really like the idea of creating new zones in under 1 second, and not wasting disk space doing it. As an added bonus, there is a chance performance will be slightly better, due to shared caching of data blocks,
I already have 2 zones setup on supernova: dns, and proxy, and tonight was time to make a start on a mail server zone to replace my suse-linux, xen based mail server.
Unfortunately it's now the end of the night, and I'm still no closer to actually setting up the new mail server, but I did at least succeed in getting a zfs cloned "mail" zone running and configured. And here it is, captured for all time
The problem was that zoneadm was simply refusing to actually clone my zone. Instead it was copying all the data across.
Here is what I started off with:
NAME USED AVAIL REFER MOUNTPOINT
Z 1.30T 567G 43.0G /Z
Z/backups 64.7G 567G 34.8K none
Z/backups/angelous 64.7G 567G 44.4G /Z/backups/angelous
Z/backups/angelous/home 20.3G 567G 1.77G /Z/backups/angelous/home
Z/backups/supernova 33.0K 567G 33.0K /Z/backups/supernova
Z/media 1.15T 567G 1.15T /Z/media
Z/storage 45.3G 567G 45.3G /Z/storage
Z/zones 1.79G 567G 36.5K none
Z/zones/dns 15.7M 567G 374M /zones/dns
Z/zones/proxy 1.70G 567G 1.69G /zones/proxy
rpool 6.66G 30.0G 36K /rpool
rpool/ROOT 5.65G 30.0G 18K legacy
rpool/ROOT/snv_95 5.65G 30.0G 5.36G /
rpool/export 66.5K 30.0G 28K /export
rpool/export/home 38.5K 30.0G 38.5K /export/home
We'll be focusing on the lines that I've bolded.
Firstly lets look at which directories map to which zfs's and for that matter, zpools.
rpool is / and most of Z is mounted under /Z/
zones however are under /zones/* which seemed logical enough to me.
I mainly use my raidz for storage of media and backups, whereas my zones as part of the core OS, are mounted directly off root as you would expect.
I wanted to make use of the 8-disk raidZ for both redundancy and performance for the zones, so putting them on the rpool was not the goal.
Z/zones was set with no mountpoint.
I did this because it doesn't actually contain any data, and since all zones will have their own zfs datasets, it will never need to contain any actual data. I've been caught out before when I've created a series of nested zfs's, and copied data into an "admin" parent directory and then "lost" it as zfs have mapped over the same namespace. Sure, in unix-speak everything just maps on top, but that's also a curse sometimes, and things can get hidden!
So, Z/zones has no mountpoint, but the children are mounted under it in /zones/dns and /zones/proxy.
Where exactly is /zones coming from, if not from Z/zones?
When I first started it was coming from / which is rpool, also a zfs fs, but the wrong zpool - which is important as I'll come to later.
I'd been running quite happily like this ever since upgrading supernova to snv_95, and dns is actually a clone of proxy - so I know that clones do work.
Yet tonight when I tried doing a zoneadm -z mail clone proxy, it was doing two things wrong. It wasn't creating a dedicated zfs for mail, and it wasn't cloning, it was copying... it even told me so.
Why? I'm running zfs root so I am sure that the prerequisite of the zonepath being on a zfs was being met!
The zonepath for mail, according to zonecfg -z mail was as follows:
zonename: mail
zonepath: /zones/mail
...which was exactly the same layout as dns and proxy.
The issue was all around the /zones directory, which technically didn't exist on my Z zpool at all, it was from /.
Now clearly zfs isn't going to be able to clone across zpools, so that explains why the clone failed. An warning/explanation would have been nice though!
But what about simply creating a zfs dataset on rpool then?
I think it wasn't doing that because zoneadm is being quite clever about it's use of naming and zfs mountpoint inheritance.
Based on my mucking around on this, it seems that zoneadm looks at the path that you've given it, and takes the parent dataset from this, and then attempts to create a child based on the path that you've provided.
I set the zonepath to /zones/mail, and I think the code then tries to access a /zones zfs. Now in my case, /zones wasn't a zfs... it was in a zfs, but it wasn't itself a zfs.
I guess zoneadm likes for zones to be directly below dedicated zfs datasets, as it can then inherit everything that it needs. One has to remember that while zfs datasets have a concept of heirachy, that doesn't actually necessarily match the logic directory layout on the disk.
In fact I do just this already with Z
The root of the Z zfs is under /Z, however /Z/zones/dns is up a directory, and then down another, which just happens to also be a zfs dataset, but until my upgrade to snv_95, it was ufs instead.
zfs and the unix mount anywhere system is almost too flexible sometimes, it gets confusing!
For zoneadm checked the provided path for a parent zfs, but because /zones wasn't a zfs, it had no way of knowing that I meant for it to create a child of Z/zones, since it wasn't mounted there.
So in the end, I was having a double failure, which results in my very weird results. At least I know understand why it was happening.
The fix was easy: I shutdown my zones, zfs umounted them, and then deleted the /zones directory from / (rpool)
With that out of the way, I set the mountpoint of Z/zones to be /zones, issued a zfs mount -a, and repeated my zonedam -z mail clone proxy.
This time around we were away, and fastforwarding a bit; here is where I'm at now:
NAME USED AVAIL REFER MOUNTPOINT
Z 1.30T 567G 43.0G /Z
Z/backups 64.7G 567G 34.8K none
Z/backups/angelous 64.7G 567G 44.4G /Z/backups/angelous
Z/backups/angelous/home 20.3G 567G 1.77G /Z/backups/angelous/home
Z/backups/supernova 33.0K 567G 33.0K /Z/backups/supernova
Z/media 1.15T 567G 1.15T /Z/media
Z/storage 45.3G 567G 45.3G /Z/storage
Z/zones 1.79G 567G 36.5K /zones
Z/zones/dns 15.7M 567G 374M /zones/dns
Z/zones/mail 83.1M 567G 1.72G /zones/mail
Z/zones/proxy 1.70G 567G 1.69G /zones/proxy
rpool 6.66G 30.0G 36K /rpool
rpool/ROOT 5.65G 30.0G 18K legacy
rpool/ROOT/snv_95 5.65G 30.0G 5.36G /
rpool/export 67.5K 30.0G 28K /export
rpool/export/home 39.5K 30.0G 39.5K /export/home
Lesson to be learned, if you wish to have zfs clone based zones, the parent directory of zonepath MUST be a zfs dataset, or zoneadm will get very confused.
First Post!!!!111
Well, it's high time that I started dumping some of this stuff out of my head.
One of the most frustrating things for me is hitting a problem and getting stuck whilst simultaneously remembering that this is both a challenge that I've faced both, and one that I've already done the hard yards solving - yet I can't remember the answer, and end up having to figure it out all over again.
I've also read from many sources that blogging is both good for the pysche (and who doesn't like a good vent now and then...and then... and then too!), and also that it improves wordpower as more practice goes into higher quality writing.
Consider this a well rounded experiment then, to see if any of this turns out to be true, and to see if I end up referring back to older posts is search of the ever elusive "ah HA!" moments.
Largely I expect this to be a private blog, without readership. I've chosen blogger.com on a whim; it was just quick and easy. Perhaps I'll self-host it in the future, although I can't imagine that there will be many benefits to be doing this, and frankly I have enough work already!
One of the most frustrating things for me is hitting a problem and getting stuck whilst simultaneously remembering that this is both a challenge that I've faced both, and one that I've already done the hard yards solving - yet I can't remember the answer, and end up having to figure it out all over again.
I've also read from many sources that blogging is both good for the pysche (and who doesn't like a good vent now and then...and then... and then too!), and also that it improves wordpower as more practice goes into higher quality writing.
Consider this a well rounded experiment then, to see if any of this turns out to be true, and to see if I end up referring back to older posts is search of the ever elusive "ah HA!" moments.
Largely I expect this to be a private blog, without readership. I've chosen blogger.com on a whim; it was just quick and easy. Perhaps I'll self-host it in the future, although I can't imagine that there will be many benefits to be doing this, and frankly I have enough work already!
Subscribe to:
Posts (Atom)