![OpenSolarisLogoXPRT[1]-SMALL.gif](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8huBlgi1aM9OmQlVpevaRBbhIjk8B_gXVGwc4ppSL8h3r3MyfeGI7bt6BksykFphUWVZBUFuCT2wphjEHKAy6KJjjk3rCs7CXHASH4nqBFdQKoFiHs0JQoQLpsHfhH9V7gHS0mHaFknA/?imgmax=800 "OpenSolarisLogoXPRT[1]-SMALL.gif")
We all know NetApp is a leading storage solution in industry. It provides reliable and performance storage array. We have been use NetApp solution for almost 2 years and never had any failure on it.
NetApp is excellent product. However it's also an expensive hardware. due to limited budget, we have to find another way to implement storage that has similar functionality but with lower cost.
In first few days of project, I have tried Openfiler, and Nexenta, but didn't make it out. Openfiler is easy to setup, but while doing CIFS share, i encountered problem in setting windows permissions. Nexenta got pretty interface. Somehow while i change NFS configuration, it didn't applied. Finally i turned to OpenSolaris with ZFS.
I have created couple storage servers by using OpenSolaris box. They are running on production environment severing storage to Oracle database and on traffic web servers. Looks pretty stable so far. Performance is around 10 ~ 30 % slower than our current NetApp storage, but cost is , you know.
Following i will share steps that how we created those OpenSolaris storage.
Share Planning First you have plan how many shares needed in each box and what type of share each share point is. in my case, there are 2 storage box. Storage I got 2 mix share. Same share goes both NFS and CIFS protocol. Storage II got 2 MIX share and 2 NFS share.
Config Network Interface Configure box DNS and running following command to refresh the change.
cp /etc/nsswitch.dns /etc/nsswitch.conf
Edit resolv.conf and add domain information in it. Ex : "domain abc.com" "search abc.hq"
Make sure default routing is going to right network interface if you have 2 or above Network Interfaces.
Edit host file (/etc/hosts) to contains all servers that need to access to it. Ex: 10.0.1.1 DB10.0.1.2 ApplicationServer-I 10.0.1.2 ApplicationServer-II
Install the Solaris CIFS service packages
Download SMB package from internet.
pkg install SUNWsmbskr
pkg install SUNWsmbs
Load the smbsrv module.
Import the Solaris CIFS service SMF manifest.
svccfg import /var/svc/manifest/network/smb/server.xml
Sync time with DNS server
Install PAM module. Add line "other password required pam_smb_passwd.so.1 nowarn" to /etc/pam.conf
Enable CIFS service
svcadm enable smb/server
You have to reboot server after you perform previous configuration.
Configure CIFS with AD
Set up Kerberos 5. fill following information in krb5.conf:
default_realm = ZZF.HQ
[realms]
ABC.HQ = {
kdc = your ad server ip or name
admin_server = your ad server
kpasswd_server = your ad server
kpasswd_protocol = SET_CHANGE
}
nano /etc/krb5/krb5.conf
SMB join ActiveDirectory
smbadm join -u administrator abc.hq
Create User Create Unix users
// make sure all UNIX system user ID group ID are same. useradd -u 999 oracle useradd -u 5501 web
Perform ID mapping on administrator and other users.
idmap add 'wingroup:administrators' 'unixgroup:sysadmin'
idmap add 'winname:administrator@zzf.hq' 'unixuser:root'
// restart service after mapping is done. i actually encountered that ID mapping doesn't always work while update mapping list.
svcadm restart smb/server; svcadm restart idmap
Setup ZFS
Create ZFS Pool
// user format to check your disk id
format
zpool create -f laketahoe c9d0
Create Shares Points
zfs create -o casesensitivity=mixed laketahoe/mixshare-I
zfs create -o casesensitivity=mixed laketahoe/mixshare-II
zfs create -o casesensitivity=mixed laketahoe/mixshare-III
zfs create -o casesensitivity=mixed laketahoe/mixshare-IV
zfs create laketahoe/nfsshare-I
zfs create laketahoe/nfsshare-II
Share
Create NFS Share
zfs set sharenfs="anon=0,rw=@192.168.1.201/24,root=@10.0.0.1/24" laketahoe/mixshare-I
zfs set sharenfs="anon=0,rw=@192.168.1.201/24,root=@10.0.0.2/24" laketahoe/mixshare-II
zfs set sharenfs="anon=0,rw=@192.168.1.201/24,root=@10.0.0.1/24" laketahoe/mixshare-III
zfs set sharenfs="anon=0,rw=@192.168.1.201/24,root=@10.0.0.2/24" laketahoe/mixshare-IV
zfs set sharenfs="anon=0,rw=@192.168.1.201/24,root=@10.0.0.1/24" laketahoe/nfsshare-I
zfs set sharenfs="anon=0,rw=@192.168.1.201/24,root=@10.0.0.2/24" laketahoe/nfsshare-II
Create CIFS Share
chmod 777 /laketahoe/mixshare-I
chmod 777 /laketahoe/mixshare-II
chmod 777 /laketahoe/mixshare-III
chmod 777 /laketahoe/mixshare-IVzfs set sharesmb=on laketahoe/mixshare-I
zfs set sharesmb=on laketahoe/mixshare-II
zfs set sharesmb=on laketahoe/mixshare-III
zfs set sharesmb=on laketahoe/mixshare-IV
check available NFS mount point. go to machine that
showmount -e [filer host name]
mount NFS
mount filer:/laketahoe/mixshare-I /mnt/mixshare
at this point , you should be able to see that NFS is mounted
df -h
you can also mount those mix share from windows box as long you use windows administrator to set right permission on those share.
those are steps are steps that make shares work. for permission setting and detail about those configuration, following are good references to go.
References
Developer Recipes: Setting Up an OpenSolaris Storage Server in 10 Minutes or Less ~ By Dominic Kay, April 29, 2008
http://developers.sun.com/openstorage/articles/opensolaris_storage_server.html
HOW to INSTALL and SHARE a File System to CIFS Clients ~ By Brian Leonard, May 22, 2009
http://www.opensolaris.com/use/CIFS.pdf
CIFS_Service_Troubleshooting ~ wiki
http://wiki.genunix.org:8080/wiki/index.php/CIFS_Service_Troubleshooting
Getting Started With the Solaris CIFS Service ~ wiki
http://wiki.genunix.org:8080/wiki/index.php/Getting_Started_With_the_Solaris_CIFS_Service
Solaris CIFS Administration Guide ~ By Sun Microsystems, Inc, March, 2009
http://dlc.sun.com/pdf/820-2429/820-2429.pdf
How to Configure the Solaris CIFS Service in Domain Mode ~ Oracle
http://docs.sun.com/app/docs/doc/820-2429/configuredomainmodetask
Encountering "Permission Deny" while mounting NFS
-> check nfs mount ip address and mask
zfs set sharenfs="anon=0,rw=@192.168.1.201/24,root=@192.168.1.201/24" laketahoe/oradatawp
"NFS compound failed for server 192.168.1.17: error 27 (RPC: Received disconnect from remote)" message shown while mounting NFS
-> check nfs mount ip address and mask
zfs set sharenfs="anon=0,rw=@192.168.1.201/24,root=@192.168.1.201/24" laketahoe/oradatawp
Encountering “nfs mount: mount: /local/httpd/htdocs: Device busy” message while mountng NFS
-> check nfs mount ip address and mask
zfs set sharenfs="anon=0,rw=@192.168.1.201/24,root=@192.168.1.201/24" laketahoe/oradatawp
Showing "nobody" in folder ownership on NFS mount
change following domain to map ZZF.HQ
#vi /etc/default/nfs
#vi /etc/resolv.conf
#vi /var/run/nfs4_domain
#svcadm restart nfs/mapid
