Ubuntu
Install
apt update
apt install zfsutils-linux
Create zpool
https://blog.programster.org/zfs-create-disk-pools
$ lsblk
striped
zpool create -f tank /dev/nvme[1-7]n1
raid z
zpool create -f tank raidz /dev/nvme[1-7]n1
Create zvol
https://cuddletech.com?p=16 thin provision -s
location
# ls /dev/zvol/tank
total 0
Get compression
# zfs get compression tank
# zfs set compression=lz4 tank
# zfs get all tank
Share iSCSI
https://docs.oracle.com/cd/E19253-01/819-5461/gaypf/index.html
https://linuxhint.com/iscsi_storage_server_ubuntu/
apt update
apt install tgt
cat /etc/tgt/conf.d/iqn.myserver.com.conf
<target iqn.2021-01.pactual.com:lun1>
backing-store /dev/zd0
initiator-address <IP>
</target>
systemctl reload tgt
tgtadm --mode target --op show
Mount iSCSI
# centos
yum -y install iscsi-initiator-utils
# ubuntu
yum -y install open-iscsi
vim /etc/iscsi/iscsi.conf
iscsiadm -m discovery -t sendtargets -p <IP>
fdisk -l
Snapshots & Clones
https://www.howtoforge.com/tutorial/how-to-use-snapshots-clones-and-replication-in-zfs-on-linux/
zfs list -t snapshot
zfs destroy tank/disk0@snap0
zfs snapshot tank/disk0@snap0
zfs clone tank/disk0@snap0 tank/disk0-clone-from-snap0
Utils
zpool detroy <label>
zpool list
Investigate Issues
Pool is unhealthy
https://www.truenas.com/community/threads/one-or-more-devices-has-experienced-an-unrecoverable-error.69236/
zpool status -v
output
pool: boot-pool
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
boot-pool ONLINE 0 0 0
da0p2 ONLINE 0 0 0
errors: No known data errors
pool: tank
state: ONLINE
status: One or more devices has experienced an unrecoverable error. An
attempt was made to correct the error. Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
using 'zpool clear' or replace the device with 'zpool replace'.
see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-9P
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
gptid/d4cdbcae-6817-11eb-a922-000c29604fa7 ONLINE 0 0 0
gptid/d8a9aae3-6817-11eb-a922-000c29604fa7 ONLINE 4 1 0
gptid/d93d3edf-6817-11eb-a922-000c29604fa7 ONLINE 0 0 0
gptid/d9311c26-6817-11eb-a922-000c29604fa7 ONLINE 0 0 0
logs
gptid/c75684e1-6817-11eb-a922-000c29604fa7 ONLINE 0 0 0
cache
gptid/c75ac75d-6817-11eb-a922-000c29604fa7 ONLINE 0 0 0
errors: No known data errors
glabel status
output
Name Status Components
iso9660/TRUENAS N/A cd0
gptid/1c2dbfeb-67ed-11eb-b853-000c29604fa7 N/A da0p1
gptid/c75684e1-6817-11eb-a922-000c29604fa7 N/A da2p1
gptid/c75ac75d-6817-11eb-a922-000c29604fa7 N/A da1p1
gptid/d4cdbcae-6817-11eb-a922-000c29604fa7 N/A da6p2
gptid/d8a9aae3-6817-11eb-a922-000c29604fa7 N/A da4p2
gptid/d9311c26-6817-11eb-a922-000c29604fa7 N/A da5p2
gptid/d93d3edf-6817-11eb-a922-000c29604fa7 N/A da3p2
gptid/d71c81c1-6817-11eb-a922-000c29604fa7 N/A da4p1
smartctl -a /dev/da4
summary
=== START OF INFORMATION SECTION ===
Model Family: Seagate Barracuda 2.5 5400
Device Model: ST5000LM000-2AN170
Serial Number: WCJ3JFBL
LU WWN Device Id: 5 000c50 0c72a0920
Firmware Version: 0001
User Capacity: 5,000,981,078,016 bytes [5.00 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: 5526 rpm
Form Factor: 2.5 inches
Device is: In smartctl database [for details use: -P show]
ATA Version is: ACS-3 T13/2161-D revision 5
SATA Version is: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Sun Feb 7 18:01:00 2021 PST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART Status not supported: Incomplete response, ATA output registers missing
SMART overall-health self-assessment test result: PASSED
Warning: This result is based on an Attribute check.
See vendor-specific Attribute list for marginal Attributes.
General SMART Values:
Offline data collection status: (0x82) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: ( 0) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 1) minutes.
Extended self-test routine
recommended polling time: ( 825) minutes.
Conveyance self-test routine
recommended polling time: ( 2) minutes.
SCT capabilities: (0x30a5) SCT Status supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000f 082 063 006 Pre-fail Always - 154052424
3 Spin_Up_Time 0x0003 098 098 000 Pre-fail Always - 0
4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 16
5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0
7 Seek_Error_Rate 0x000f 078 060 045 Pre-fail Always - 63321072
9 Power_On_Hours 0x0032 100 100 000 Old_age Always - 430 (60 120 0)
10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0
12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 13
183 SATA_Downshift_Count 0x0032 100 100 000 Old_age Always - 0
184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0
187 Reported_Uncorrect 0x0032 099 099 000 Old_age Always - 1
188 Command_Timeout 0x0032 100 099 000 Old_age Always - 4295032834
189 High_Fly_Writes 0x003a 100 100 000 Old_age Always - 0
190 Airflow_Temperature_Cel 0x0022 040 030 040 Old_age Always FAILING_NOW 60 (Min/Max 38/70 #1688)
191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0
192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 74
193 Load_Cycle_Count 0x0032 100 100 000 Old_age Always - 243
194 Temperature_Celsius 0x0022 060 070 000 Old_age Always - 60 (0 25 0 0 0)
195 Hardware_ECC_Recovered 0x001a 082 064 000 Old_age Always - 154052424
197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0
240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 376 (151 183 0)
241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 6158404140
242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 59090988
254 Free_Fall_Sensor 0x0032 100 100 000 Old_age Always - 0
``