Hot-Swap with Intel RAID (Rapid Storage Technology – RST/RSTe)

It is 100% possible to hot-swap with Intel RAID when it is enabled and configure with an array of drives. As in, just pull the drive and put a new drive in while the OS is still running. There is not an option to configure it to automatically rebuild the array however. The rebuild option is only available in the Windows utility.

  1. Pull the drive to be replace.
  2. Push in new drive.
  3. In Intel RSTe utility, select the volume and click “Rebuild” on the right panel.

Rebuilding a 4x 1TB array took about 4 hours.

Posted in Uncategorized | Leave a comment

Ubuntu Hot-Swap MDADM Array and Extend/Grow

Step-by-step guide

  1. Mark the drive as failed
    1
    $ sudo mdadm --manage /dev/md0 --fail /dev/sdb1
  2. Remove the drive from the array
    1
    $ sudo mdadm --manage /dev/md0 --remove /dev/sdb1
  3. View the mdadm status
    1
    $ cat /proc/mdstat
  4. If you prefer to shut down the system for a cold swap, do it now. Before the hot swap, put the drive into standby with the following command
    1
    $ sudo hdparm -Y /dev/sdb

    Make sure you know which drive you are going to remove before issuing this command. Operations to the disk will wake up the drive again.
  5. Remove the SATA signal cable first and then the SATA power cable.
  6. Mount the new drive and connect SATA power. I let the drive spin up for 5-10 seconds before connecting the SATA signal cable. If you did a cold swap, power on the system at this point.
  7. Identify the new drive and what device name it has. In my case, the new drive was conveniently named /dev/sdb, the same as the old one.
  8. Copy the partitioning setup from the other drive in the array to the new disk. (This is for a RAID1 setup)
    Make sure the order is correct, otherwise we will erase the operational drive!
    1
    $ sfdisk -d /dev/sdc | sfdisk /dev/sdb
  9. Add the new drive to the RAID array
    1
    $ sudo mdadm --manage /dev/md0 --add /dev/sdb1
  10. The RAID array will now be rebuilt and the progress is indicated by the
    1
    $ cat /proc/mdstat

    output. To have a more dynamic update of the progress use the following:
    1
    $ watch cat /proc/mdstat
Posted in Uncategorized | Leave a comment

LXD Guest Migration to New Host

LXD makes it easy to move containers around. LXD expects the new host to be completely setup along with the resources the guest requires or the copy will not proceed.

1
2
3
# destination
lxc config set core.https_address 192.168.0.1:8443
lxc config set core.trust_password YourPassword
1
2
3
4
# source
lxc remote add NewHost 19.168.0.1:8443
lxc copy ContainerName NewHost:ContainerName --mode="push"
# WARNING: do not ctrl-c the copy it will mess up the host LXD and must be reinstalled.
Posted in Uncategorized | Leave a comment

Fix Ubuntu Grub Boot Error

This error happens usually when the /boot folder is full and grub doesn’t update/install properly when the system is updated.

1
2
3
error: symbol 'grub_file_filters' not found.
Entering rescue mode...
grub rescue>

One way to fix this is to boot with a Live USB image, mount the boot drive, and update/install grub again.

1
2
3
4
5
6
7
mount /dev/<vg/root> /mnt
mount -t proc none /mnt/proc
mount -o bind /dev /mnt/dev
mount -t sysfs /sys /mnt/sys
chroot /mnt
grub-install /dev/sda
update-grub
Posted in Uncategorized | Leave a comment

Install Windows 10 via Linux PXE

Windows 10 setup does *not* work via PXE ram disk when the iso is directly loaded. A Pre-Environment must be loaded first for to load setup from a mapped network share where the iso image has been dumped.

This guide assumes some prerequisites:
1. working PXE environment
2. pre-built Windows PE image
3. Samba/Windows share with the dumped Windows 10 image

pxelinux config file should contain the following bit to load Windows PE.

1
2
3
label Windows PE x64
  kernel memdisk
  append iso initrd=images/WinPE_x64.iso

PE will load into a command prompt.

1
2
3
initpe
net use * \\192.168.0.220\tftpboot\images\win10
z:\setup

Make sure the shared directory has read and execute rights; otherwise, it will show “Access is denied”.

Posted in Uncategorized | Leave a comment

Analysis of the Intel AXXRMM4LITE iKVM Module

Intel line of server motherboards have iKVM built-in but it must be enabled with a separate licensing module that plugs into the motherboard. I got one recently and was curious as to how this “license” works.

For the AXXRMM4LITE module, it is a single SPI flash (Winbond 25X10CLNIG) with an 8-pin connector. To my surprise, the flash is completely empty. The BMC/BIOS only checks for the presence of an SPI fash.

SPI flash
8-pin connector pin-out
Posted in Uncategorized | 3 Comments

JIRA Bulk Link via API

JIRA can’t natively perform bulk actions on issue links. I needed to move some links to a different link type. Here’s a quick script in Javascript that runs with NodeJS.

It querys for all the relevant issues using JQL and then iterates over each issue and link that matches the link type.

'use strict';

const request = require('request');
const async = require('async');

const user = 'youruser';
const pass = 'yourpass';
const url = 'http://yourjira';
const api = url + '/rest/api/2/';

// find all issues of type 'work package'
request({
  auth: {
    user: user,
    pass: pass
  },
  url: api + 'search',
  method: 'post',
  json: true,
  body: {
    jql: 'project = TEST AND issuetype = "Work Package"',
    maxResults: 100,
    fields: ['issuelinks']
  }
}, (err, res, body) => {
  // console.log(JSON.stringify(body, null, 4));

  // for each link type of acceptance criteria, delete, relink as '
  async.eachSeries(body.issues, (issue, doneIssue) => {
    console.log('processing ' + issue.key);
    async.eachSeries(issue.fields.issuelinks, (link, doneLink) => {
      if (link.outwardIssue && link.outwardIssue.fields.issuetype.name === 'Acceptance Criteria' && link.type.name === 'Contains') {
        console.log('processing link '+ link.id);
        async.series([
          function (done) {
            request({
              auth: {
                user: user,
                pass: pass
              },
              url: api + 'issueLink' + '/' + link.id,
              method: 'delete',
              json: true
            }, done);
          },
          function (done) {
            request({
              auth: {
                user: user,
                pass: pass
              },
              url: api + 'issueLink',
              method: 'post',
              json: true,
              body: {
                type: {
                  name: 'Requirements'
                },
                inwardIssue: {
                  'key': issue.key
                },
                outwardIssue: {
                  'key': link.outwardIssue.key
                }
              }
            }, done);
          }
        ], doneLink);
      } else {
        doneLink();
      }
    }, doneIssue);
  }, (err) => {
    console.log('complete');
  });
});

Posted in Uncategorized | Tagged , | Leave a comment

Zabbix with Sendgrid SMTP Notification on Ubuntu

The built-in mail option seems to work out of the box, but every post seems to suggest using a script to trigger internal mail binaries for mail notifications. Those require extra dependencies and configuration. With SendGrid and the REST API, it can be simply done with a script using curl. I assume a SendGrid account and key have already been setup.

Creating the script

sudo vim /usr/lib/zabbix/alertscripts/sendgrid.sh

sendgrid.sh

#!/bin/bash
SENDGRID_API_KEY="YOURKEYHERE"

curl --request POST \
 --url "https://api.sendgrid.com/v3/mail/send" \
 --header "Authorization: Bearer $SENDGRID_API_KEY" \
 --header 'Content-Type: application/json' \
 --data "{\"personalizations\": [{\"to\": [{
\"email\": \"$1\"}]}],\"from\": {\"email\": \"[email protected]\"},\"subject\": \"$2\",\"content\": [{\"type\": \"text/plain\", \"value\": \"$3\"}]}"

Notification Testing

There is no way to test aside from triggering an actual fault, so it’s necessary to create a dummy condition and then trigger it with the zabbix_sender utility. I had to explicitly install it:

Install zabbix_sender

sudo apt install zabbix_sender

Create Action and Condition

  1. Configuration -> Actions -> Create action
  2. Select condition
  3. Add new condition (= Dummy trigger)
  4. Select the Operations tab
  5. Add new operations (user with custom media type)
  6. Save by clicking Add

Trigger

zabbix_sender --zabbix-server=127.0.0.1 --host="192.168.0.240" --key="test.timestamp" --value="${VALUE}"

De-trigger

VALUE="$(date --rfc-3339=ns)"; zabbix_sender --zabbix-server=127.0.0.1 --host="192.168.0.240" --key="test.timestamp" --value="${VALUE}"

reference: http://cavaliercoder.com/blog/testing-zabbix-actions.html

 

Posted in Uncategorized | Leave a comment

Installing oracle-java9-installer on Ubuntu Error Fix

The oracle-java9-installer, as of writing, has an old URL that doesn’t redirect properly to the right URL, which causes the installer to fail when it tries to download the binaries.

Setting up oracle-java9-installer (9b162-1~webupd8~0) ...
Using wget settings from /var/cache/oracle-jdk9-installer/wgetrc
Downloading Oracle Java 9...
--2017-05-19 04:10:54-- http://www.java.net/download/java/jdk9/archive/162/binaries/jdk-9-ea+162_linux-x64_bin.tar.gz
Resolving www.java.net (www.java.net)... 137.254.56.25
Connecting to www.java.net (www.java.net)|137.254.56.25|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://home.java.net/download/java/jdk9/archive/162/binaries/jdk-9-ea+162_linux-x64_bin.tar.gz [following]
--2017-05-19 04:10:54-- https://home.java.net/download/java/jdk9/archive/162/binaries/jdk-9-ea+162_linux-x64_bin.tar.gz
Resolving home.java.net (home.java.net)... 156.151.59.19
Connecting to home.java.net (home.java.net)|156.151.59.19|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://www.oracle.com/splash/java.net/maintenance/index.html [following]
--2017-05-19 04:10:54-- http://www.oracle.com/splash/java.net/maintenance/index.html
Resolving www.oracle.com (www.oracle.com)... 184.30.70.138, 2600:1408:10:184::2d3e, 2600:1408:10:185::2d3e
Connecting to www.oracle.com (www.oracle.com)|184.30.70.138|:80... connected.
HTTP request sent, awaiting response... 503 Service Unavailable
2017-05-19 04:10:54 ERROR 503: Service Unavailable.

download failed
Oracle JDK 9 is NOT installed.

You will need to manually install the binary and run dpkg to configure it. Change the URL from http://www.java.net/download/ to http://download.java.net/.

cd /var/cache/oracle-jdk9-installer
sudo wget http://download.java.net/java/jdk9/archive/162/binaries/jdk-9-ea+162_linux-x64_bin.tar.gz
sudo dpkg --configure -a
Posted in Uncategorized | 3 Comments

Resizing Virtualbox Fixed-Size VDI Disks

As of writing (2017-04-06), there is no native tooling to resize a fixed-size VDI. Those on the internet saying they can resize one are misinformed. You will get this error message:

Progress state: VBOX_E_NOT_SUPPORTED
VBoxManage.exe: error: Resize medium operation for this format is not implemented yet!

To “resize” a fixed-size VDI, it must be cloned to a larger sized VDI.

Step 1 – Create the larger VDI and move data:

Method 1 – Using VBoxManage:

VBoxManage clonehd [old-VDI] [new-VDI] --variant Standard
VBoxManage modifyhd [VDI] --resize [megabytes]
VBoxManage clonehd [new-VDI] [newnew-VDI] --variant Fixed

Disadvantage of this method is you need to make two additional full copies of the disk.

Method 2 – Using Clonezilla:

  1. Create and attach a new, larger fixed-size VDI using the VirtualBox interface.
  2. Attach and boot with a Clonezilla ISO.
  3. Use Device-to-device setting to clone the drive. Remember to press F12 on boot to select CD-ROM.

Step 2 – Expand the underlying partitions:

  1. De-attach Clonezilla ISO and attach GParted ISO.
  2. Boot similarly and resize/move partitions as needed.
Posted in Uncategorized | 2 Comments