megacli basic usage

by Pedro M. S. Oliveira | Aug 20, 2014 | Linux

When you have large deployments with thousands of SSDs and spinning disks megacli utility provides a great help by having all the features and options available in a way that can be easily scripted and therefore automated.
This usage listing just has the most used and therefore limited set of options, but many more exist, please check the bottom for references.

Here are a few of those I find more useful, this will only apply to LSI Raid controllers.

Info:

This will list all the physical devices on adapter 0, if you have more than 1 controller in your server you may use -aAll.

megacli -PDList -a0

The same as above but it will be easy to find if you have 1 slot with errors.

megacli -PDList -a0 | grep “Slot\|Error”

This will display all the settings for the logical device 1 on controller 0.

megacli -LDInfo -L1 -a0

It will display the Consistency of the logical device 1 on controller 0.

megacli -LDGetProp Consistency -L1 -a0

Show rebuild progress on logical device 2 on adapter 0.

megacli -LDRecon ShowProg L2 -a0

List the cache status on adapter 0.

megacli -GetPreservedCacheList -a0

Display auto rebuild state on adapter 0.

megacli -AdpAutoRbld -Dsply -a0

Display missing physical devices on controller 0.

megacli -PdGetMissing -a0

Create a file called megacli_events_since_reboot that will contain all the events logged by all the controllers, this will include warnings, info messages and errors since last reboot.

megacli -AdpEventLog -GetSinceReboot -f megacli_events_since_reboot -aALL

Create a file called megacli_events_since_shutdown that will contain all the events logged by all the controllers, this will include warnings, info messages and errors since last shutdown.

megacli -AdpEventLog -GetSinceShutdown -f megacli_events_since_shutdown -aALL

Show the rebuild progress for the drive in slot 21, enclosure 32, all adapters (you can use -a0 if you just have one adapter).

megacli -pdrbld -showprog -physdrv[32:21] -aALL

Add/alter settings:

Set rebuild rate to 60% (1-100), this will mean that rebuild has higher priority than SO calls.

megacli -AdpSetProp RebuildRate 60 -a0

Discard the preserved cache for all logical devices on adapter 0.

megacli -DiscardPreservedCache -Lall -a0

Turn off device on slot 2 on enclosure 32, adapter 0.

megacli -PDOffline -PhysDrv [32:2] -a0

Turn on device on slot 2, enclosure 32, adapter 0.

megacli -PDOnline -PhysDrv [32:2] -a0

Flag the the device on slot 2, enclosure 32, adapter 0 as good to be used.

megacli -PDMakeGood -PhysDrv[32:2] -a0

Mark as missing drive on slote 2, enclosure 32 adapter 0

MegaCli -PDMarkMissing -PhysDrv [32:2] -a0

Prepare for removal drive on slote 2, enclosure 32 adapter 0

MegaCli -PdPrpRmv -PhysDrv [32:2] -a0

Replace the missing physical drive on slot 2, enclosure 32 on array 2, row 2, adapter 0.

You may find more information on the missing drive with the option -PdGetMissing (explained above).

megacli -PdReplaceMissing -physdrv [32:2] -Array=2 -row=2 -a0

Initial rebuild on slot 2, enclosure 32, adapter 0.

megacli -PDRbld -Start -PhysDrv[32:2] -a0

Create a logical device with raid 0 on physical device on slot 5, enclosure 32, adapter 0.

megacli -CfgLdAdd -r0[32:5] -a0

Set a dedicated hotspare device on logical device 1 (here it’s called array) using device on slot 18, on enclosure 32, adapter 0.

megacli -PDHSP -Set -Dedicated -Array1 -PhysDrv[32:18] -a0

Remove hotspare located on slot 6, enclosure 32, adapter 0.

megacli -PDHSP -Rmv -PhysDrv[32:6] -a0

Make drive on slot 18, enclosure 32, adapter 0 offline.

megacli -PDOffline -PhysDrv [32:18] -a0

Add 8 drives to an existing raid 6, in this case we are adding it to the logical volume 2 on adapter 0.

megacli -LDRecon start r6 -Add-PhysDrv[32:14,32:15,32:16,32:17,32:18,32:19,32:20,32:21] L2 -a0

Drive firmware update procedure:

* firmware upgrade can brick your device, first make your drive offline, this is MANDATORY if the drive is online

Make drive on slot 18, enclosure 32, adapter 0 offline

megacli -PDOffline -PhysDrv [1:18] -a0

Update firmware on drive at slot 18, enclosure 32 adapter 0, with binary file called fw.binary

megacli -PdFwDownload -PhysDrv[32:18] -f fw.bin -a0

Put drive back online.

megacli -PDOnline -PhysDrv [32:18] -a0

OS rescan logical device

After adding/removing/editing the logical volume, please note that you need to do this to all block devices you changed, in this case /dev/sda

echo 1 > /sys/block/sdx/device/rescan

More information:

http://hwraid.le-vert.net/wiki/LSIMegaRAIDSAS
http://mycusthelp.info/LSI/_cs/AnswerPreview.aspx?sSessionID=&inc=8275
http://artipc10.vub.ac.be/wordpress/2011/09/12/megacli-useful-commands/

Megacli official manual (PDF)

Cheers,

Pedro M. S. Oliveira

Listing storage (scsi) paths for use with multipath

by Pedro M. S. Oliveira | Mar 9, 2012 | Linux

Hi!

About a year ago I’ve setup some linux RHEL 6 with multipath access to an HP EVA storage.

Today I needed to do it again, and to do so i needed to list all the path available to the storage device, here’s my command line (hope it helps someone else) to list all the path and volumes:


ls /dev/sd* | grep -E -v '[0-9]' | while read D ; do F=$(echo $D) ; \
echo -n $F ; echo -n " " ; scsi_id --page=0x83 --whitelisted --device=$F | \
sort -k 2 ; done | sort -k 2

The output should be something like this:


/dev/sda 3600508b1001c927a634cedb90322b49e
/dev/sdb 3600508b4000744ff0000a00001fd0000
/dev/sdf 3600508b4000744ff0000a00001fd0000
/dev/sdj 3600508b4000744ff0000a00001fd0000
/dev/sdn 3600508b4000744ff0000a00001fd0000
/dev/sdd 3600508b4000744ff0000a000025c0000
/dev/sdh 3600508b4000744ff0000a000025c0000
/dev/sdl 3600508b4000744ff0000a000025c0000
/dev/sdp 3600508b4000744ff0000a000025c0000
/dev/sde 3600508b4000744ff0000a000025f0000
/dev/sdi 3600508b4000744ff0000a000025f0000
/dev/sdm 3600508b4000744ff0000a000025f0000
/dev/sdq 3600508b4000744ff0000a000025f0000
/dev/sdc 3600508b4000744ff0000a00002660000
/dev/sdg 3600508b4000744ff0000a00002660000
/dev/sdk 3600508b4000744ff0000a00002660000
/dev/sdo 3600508b4000744ff0000a00002660000

As you can see I’ve one available disk on this server, actually this one is a RAID1 (HW) config (sda), 16 paths to my storage device, that delivers 4 different volumes (4 paths to each volume).
Sometime later I’ll discuss the multipath configuration but for now i just wanted to leave the command line that help me list all the paths ids.
Cheers,
Pedro Oliveira