Info Fast ZFS Storageserver with Oracle Solaris, OmniOS and napp-it

[gea]

Bryan Cantrill comments about Solaris/ Oracle ( https://en.wikipedia.org/wiki/Bryan_Cantrill )
http://dtrace.org/blogs/bmc/2017/09/04/the-sudden-death-and-eternal-life-of-solaris/

 

[Elixer]

Sigh.

RIP Solaris.
I wonder if OpenSolaris will rise from the ashes again?

 

[gea]

OpenSolaris is not only the base of Oracle Solaris 11 but was forked in 2010 to the OpenSource Illumos project. So we do have a free Solaris that is since developped completely independent from Oracle and distributions based on the common Illumos project like OpenIndiana (quite a sucessor of OpenSolaris with an optional Mate GUI), OmniOS as a strong storage server option with stables, SmartOS from Joyent/Samsung as a Cloud/VM option and NexentaStor as a commercial storage server - all based on Illumos.

Beside Encryption and sequential resilvering (avaiable on Solaris and on the way to Open-ZFS) there is now more development for ZFS outside of Oracle.

Beside that, the strategy of Oracle is not completely clear.
They now offer new servers with Sparc M8 and Solaris support until at least 2034
Oracle softly increments SPARC M7 to M8, then whispers: We'll still love you, Solaris, to 2034

What seems clear is that Oracle reduces hardware/ server offers and Solaris as an offer for non-Oracle hardware or X86 hardware. For x86 we gladly have OpenSource Solaris with a development that is completely independent from Oracle. If you want Oracle Solaris you must propably buy Oracle servers or check their (future) Cloud options.

Oracle Roadmap Sep/2017
http://www.oracle.com/us/products/s.../oracle-sparc/sparc-roadmap-slide-2076743.pdf

 

[gea]

Update

There is a new OpenIndiana 2017.10
Release notes: https://wiki.openindiana.org/oi/2017.10+Release+notes

There is a new OmniOS Community Edition r151024 stable
Release Notes: https://github.com/omniosorg/omnios-build/blob/r151024/doc/ReleaseNotes.md

I have added an option to patch sas3ircu forr Illumos
See menu Disks > Disks Location > patch sas3ircu
You need the patch especially for the new Avago/LSI 9305 HBA

see illumos issues 6784

 

[gea]

ZFS is crash resistent!

This means that ZFS - a Copy on Write Filesystem is not corrupt after a crash during writes (unlike older filesystems). This does not mean that a VM or a database is consistent on a crash as ZFS uses up to 4 GB RAM as writecache for a better performance. Think of an accounting software where you put off an amount from one account and the system crashes prior you can add it to another account (money in data nirwana as the ramcache is lost on a crash).

ZFS offers sync write, a mechanism where every commited write is logged. On a crash the commited writes are done on next reboot to allow a database or VM to be consistent. Sadly sync write requires a log device with powerloss protection and ultra low latency for a good performance. In the past these log devices (slog) were expensive and despire slow compared to fast writes without sync.

The new Intel Optane is a game-changing technology. When you use them as an Slog, even sequential sync writes are nearly as fast as writes without sync. If you use Optane not for Slog but the pool itself, it opens a new performance level on small random writes and sequential writes. Even a filer with sync enabled is possible now. I am impressed!

See http://napp-it.org/doc/downloads/optane_slog_pool_performane.pdf

 

[gea]

Bad news

Intel suddenly removed the 900P from the list of PLP protected disks and lowers the price.
https://ark.intel.com/Search/FeatureFilter?productType=solidstatedrives&EPLDP=true
so the Optane with powerloss protection at the moment is reduced to the 4800X

 

[gea]

I have made some benchmarks to answer

- napp-it AiO or barebone setup
- How important is RAM for ZFS (2GB vs 4GB vs 8GB vs 16GB vs 24GB)
- Differences between HD, SSD, NVMe Flash, NVMe Optane 900P and ZFS scaling over number of vdevs
- What about Slog (ZeusRAM vs Intel DC 3700 vs P3600 vs Optane 900P)

http://napp-it.org/doc/downloads/optane_slog_pool_performane.pdf

 

[gea]

Oracle Solaris 11.4 Public Beta released
https://blogs.oracle.com/solaris/oracle-solaris-114-beta-released

Main Features
SMB3, NFS 4.1 and dedup2
(ZFS v.43, see ZFS Pool Versions - Managing ZFS File Systems in Oracle® Solaris 11.4)

What's new
https://docs.oracle.com/cd/E37838_01/html/E60974/index.html

Datasheet
http://www.oracle.com/technetwork/server-storage/solaris11/114beta/114betadatasheet-4260629.pdf

Download
http://www.oracle.com/technetwork/server-storage/solaris11/114beta/solaris114beta-4257760.html

 

[gea]

Update

napp-it is running from current release Feb 02 on Solaris 11.4b (not all functions tested)
If you want that the napp-it wget installer compiles ex smartmontools 6.6 you should
set the beta repository prior napp-it and install gcc (pkg install gcc-5)

You need to setup the beta repository. If you have defined it after a napp-it setup,
install storage services manually
pkg install --accept --deny-new-be storage/storage-server

 

[gea]

Commercial support by the OmniOS developers is available again
https://www.omniosce.org/invoice

 

[gea]

ZFS design principles

As I often need to answer similar questions, I have collected my view in
http://napp-it.org/doc/downloads/zfs_design_principles.pdf

Remarks, infos about typo (or bad style, I am not a native speaker) or suggestions are welcome

 

[gea]

new in napp-it 18.06 Dev (Apr 11)

to be prepared for next OmniOS 151026 stable in may (or current bloody)
https://github.com/omniosorg/omnios-build/blob/r151026/doc/ReleaseNotes.md

Support for vdev removal (new ZFS feature) in menu Pools > Shrink Pool (OmniOS 151025+)
Support for poolwide checkpoints (new ZFS feature) in menu Snaps > Checkpoint (OmniOS 151025+) and Pool > Import

Disk Detection: adds ATTO HBAs (ATTO, a media specialist now supports Illumos)
Disk Map: correct detection of HBA even if disks on different LSI mptsas HBAs shows the same c-number ex c0t500..
Disk Map: add dd disk detection
Disk Location: dd detection for all disks

 

[gea]

The new OmniOS is the first Open-ZFS storage distribution to include the vdev remove (Pool shrink).
Oracle Solaris 11.4 also comes with this feature but it seems with less restriction.

Open-ZFS, at least currently lacks the support for a vdev remove of a basic or mirror vdev when a raid-z [1-3] vdev is part of the pool or a remove raid-Z[1-3] at all or add a raid-z [1-3] after a remove of ex a basic/mirror vdev what limits its use cases. Support of raid-Z [2-3] is expected in Open-ZFS (but not Z1),

Open-ZFS ex OmniOS that is the first to include this feature also requires a re-mapping table with a continous small RAM need/reservation and small performance degration.This is listed in the output of a zpool status. A manual zpool remap can fix this.

It seems that Solaris 11.4 does not have these restrictions
vdev removal, poolwide checkpoints/snaps or imp... | Oracle Community

 

[gea]

Oracle Solaris 11.4 Beta Refresh is now available:
https://community.oracle.com/message/14784683#14784683

This refresh includes new capabilities and additional bug fixes.
Some new features in this release:

• ZFS Device Removal
• ZFS Scheduled Scrub
• SMB 3.1.1

 

[gea]

A new snapshot of Openindiana Hipster 2018.04 is available

OpenIndiana Hipster is a rolling distribution of the opensource Solaris fork Illumos with a snapshot every 6 months. It comes in the three flavours GUI with a Mate desktop, Text (very similar to OmniOS, another Illumos distribution) and Minimal.

https://wiki.openindiana.org/oi/2018.04+Release+notes

 

[gea]

OmniOS 151026 (may.07.2018) is out

Release note: https://github.com/omniosorg/omnios-build/blob/r151026/doc/ReleaseNotes.md
Download: https://downloads.omniosce.org/media/r151026/

Main improvements:

- Protection against the Meltdown Intel CPU vulnerability announced earlier this year
- Experimental support for bhyve - a fast, lightweight and modern hypervisor
- Sparse-branded zones, clocking in under 4MB per zone
- An improved Installer which is dramatically faster making the OmniOS installation procedure
one of the fastest in the industry. The new installer also provides many more options for
customising the installed system.
- A new lightweight default MTA (Dragonfly Mail Agent)
- Fault management improvements for SSD disks

ZFS features
- Improved support for ZFS pool recovery (import good data from a damaged pool)
- The new zfs remove of toplevel vdevs
- zfs checkpoint features (poolwide checkpoint to make even a fs destroy/ vdev add/remove undoable)
- Support for raidz2 and raidz3 boot disks

Hardware
- support of the new BroadCom tri-mode HBAs

napp-it supports OmniOS 151026 up from 18.01 (apr.02)

 

[Hi-Fi Man]

What's the state of Illumos/OpenIndiana right now? I know a lot of Solaris developers apparently got laid off; are any of them helping out with these Solarish projects?

I ask because I just setup a simple backup server and was trying to decide what OS to use. I actually ended up going with OpenIndiana for a few reasons. For starters I ruled out Linux due to Openmediavault having some system breaking bugs. So I decided to take a look at Unix and that's when I found out about OpenIndiana and how it's OpenZFS and SMB implementation were supposedly superior to FreeBSD's. So far the system has given me no issues along with napp-it and only a few minor complaints, lack of a temperature readout for x86 being one and the interface of napp-it being another. Napp-it definitely does the job but the design of the interface is much like a German car, not as intuitive or simple as it should be.

For now I'll stick with OpenIndiana but it would be nice to know what the future holds for Solaris/Solarish in general.

 

[gea]

When Oracle took over Sun in 2010, they closed the OpenSource project OpenSolaris and published the closed source commercial OS Solaris 11 out of OpenSolaris.

OpenSolaris was then forked in the Illumos project as a common effort of all firms that had build products ontop of OpenSolaris like Delphix, Nexenta, OmniTi or Joyent. OpenIndiana was and is the community project to continue the idea of OpenSolaris.

OpenIndiana is a general use server OS with a desktop option, just like its predecessor OpenSolaris with a repository that contains a lot of apps, even desktop ones. It is based on ongoing pure Illumos (the common development roof of all OpenSource Opensolaris derived systems). Current OI is from may 2018.

Tha main alternative, OmniOS is a very minimalistic OpenSource server OS for production grade storage. The OS and the repository is stripped down to the use case FC/iSCSI/NFS/SMB/VM server to be as stable as possible. Every OmniOS (bloody, stable and long term stable) comes with its own repository (a freeze of Illumos with parts of the SmartOS fork for LX/bhyve virtualisation) with backports of critical bugfixes. A stable comes every 6 months, a long term stable every two years. For production use you can add a support contract with the developers (two firms in Europe with the infrastructure in Swiss/ETH Zürich). Current stable is from may 2018.

To extend both repositories for other applications, you can use pkgsrc from SmartOS/Joyent
https://pkgsrc.joyent.com/

Software for all Illumos based systems, see https://pkgsrc.joyent.com/packages/SmartOS/

 

[gea]

Info

There is a refresh of the Solaris 11.4 beta
https://blogs.oracle.com/solaris/oracle-solaris-114-open-beta-refresh-2

 

[gea]

Midnight Commander
Midnight commander (mc) is a filebrowser that allows browsing/ copying/ moving/ viewing and editing files from a text console.

As mc is not in the current Solaris 11.4 or OmniOS repository the fastest way to get it is my online installer:

If midnight commander does not show correct borders in Putty:
- open Putty settings Window > Translation: modify ex from UTF-8 to ISO 8859-1 (Latin-1, West Europe)
- reconnect


You cannot link to your own website.


esquared
Anandtech Forum Director

 

[gea]

Oracle Solaris 11.4 Final is ready
Oracle Solaris 11.4: Secure. Simple. Cloud Ready

 

[gea]

AiO with Solaris 11.4 on ESXi 6.7
No support from current Vmware tools for Solaris 11.4 final from today
vmware-tools / open-vm-tools on 11.4b |Oracle Community

My findings/ "just a hack"

VMware vmtools for Solaris from ESXi 6.7
Executing on a textonly setup of S11.4 final on ESXi 6.7

Installer vmware-install-pl on 11.4 installs but fails with a message
Package "SUNWuiu8" not found when executing vmtool/bin/vmware-config-tools.pl

This can be skipped by editing vmtool/bin/vmware-config-tools.pl in line 13026
when you comment out the check for SUNWuiu8

When you then run vmtool\bin\vmware-config-tools.pl it hangs due a missing /usr/bin/isalist
I copied isalist over from a Solaris 11.3, made it executable and then vmware-config-tools.pl works

After a reboot I got the message vmools installed with a console message
Warning: Signature verification of module /kernel/drv/amd64/vmmemctl failed

same with verification of the vmxnet3s driver
vmxnet3s reports deprecated "misc/mac"

Not sure if this is critical

vmxnet3s and guest restart from ESXi works

Gea

 

[gea]

Full OS/ Appliance disaster recovery

On current napp-it 18.09dev I have added a new function/menu: System > Recovery
with modified replication jobs to make recovery easy. The idea behind is:

To recover a fully configured appliance from a BE (bootenvironment):

1. backup current BE: create a replication job (require 18.09dev) with current BE as source

2. reinstall OS and napp-it

3. restore BE: create a replication job with the BE as source and rpool/ROOT as target (require 18.09dev)

4. activate the restored BE and reboot

This BE backup/restore can be done also manually via replication and zfs send

 

[gea]

OpenIndiana 2018.10 is available

Release notes and download
https://wiki.openindiana.org/oi/2018.10+Release+notes

 

[gea]

OmniOS 151028 is available since today
https://github.com/omniosorg/omnios-build/blob/r151028/doc/ReleaseNotes.md

OmniOS 151028 comes with a newer Perl and SSH/sudo handling
After an update, you may get permission or io.tty errors.

To avoid these problems, update napp-it Free/Pro/Dev to newest prior an OmniOS update.