Archivos de la categoría ‘linux/unix’

These is one of the proposed solutions for the job assessment commented in a previous post.

How could you backup all the data of a MySQL installation without taking the DB down at all. The DB is 100G and is write intensive. Provide script.

My solution to this problem will be “Online backup using LVM snapshots in a replication slave”.
It implies that a series of previous decisions have been taken:

  • LVM snapshots: Database data files must be on a LVM volume, which will allow the creation of snapshots. An alternative could be use Btrfs snapshots.

    Advantages:

    • Almost online backup: The DB will continue working while copying the data files.
    • Simple to setup and without cost.
    • You can even start a separated mysql instance (with RW snapshots in LVM2) to perform any task.

    Drawbacks:

    • To ensure the data file integrity the tables must be locked while creating the snapshot.
      The snapshot creation is fast (~2s), but this means a lock anyway.
    • All datafiles must be in the same Logical Volume, to ensure an atomic snapshot.
    • The snapshot has an overhead that will decrease the write/read performance
      (it is said
      that up to a 6x overhead). This is because any Logical Extend modified must be copied. After a while this overhead will be reduced because changes are usually made in the same Logical Extends.
  • Replication slaves: (see official documentation about replication backups and backup raw data in slaves ).
    Backup operations will be executed in a slave instead of in the master.

    Advantages:

    • Will avoid the performance impact in the Master mysql server,
      since “the slave can be paused and shut down without affecting the running operation of the master”.
    • Any backup technique can be done. In fact, using this second method should be enough.
    • Simple to setup.
    • If there is already a MySQL cluster it will use existent infrastructure.

    Drawbacks:

    • This is more an architectonic solution or backup policy than a backupscript.
    • If there are logical inconsistencies in the slave, they are included in the backup.

So, I will suppose that:

  • MySQL data files are stored in a LVM logical volume, with enough free
    LEs in the volume group to create snapshots.
  • The backupscript will be executed in a working replication slave, not in the master.
    Execution of this script on a master will result in a short service pause (tables locked)
    and I/O performance impact.

This is the backup script (tested with xfs) that must be executed on a Slave replication server:

(in github)

#!/usr/bin/python
# -*- coding: utf-8 -*-
#
# Requirements
# - Data files must be in lvm
# - Optionally in xfs (xfs_freeze)
# - User must have LOCK TABLES and RELOAD privilieges::
#
#    grant LOCK TABLES, RELOAD on *.*
#        to backupuser@localhost
#        identified by 'backupassword';
#
import MySQLdb
import sys
import os
from datetime import datetime

# DB Configuration
MYSQL_HOST = "localhost" # Where the slave is
MYSQL_PORT = 3306
MYSQL_USER = "backupuser"
MYSQL_PASSWD = "backupassword"
MYSQL_DB = "appdb"

# Datafiles location and LVM information
DATA_FILES_PATH = "/mysql/data" # do not add / at the end
DATA_FILES_LV = "/dev/datavg/datalv"
SNAPSHOT_SIZE = "10G" # tune de size as needed.

SNAPSHOT_MOUNTPOINT = "/mysql/data.snapshot" # do not add / at the end

# Backup target conf
BACKUP_DESTINATION = "/mysql/data.backup"

#----------------------------------------------------------------
# Commands
# Avoids sudo ask the password
#SUDO = "SUDO_ASKPASS=/bin/true /usr/bin/sudo -A "
SUDO = "sudo"
LVCREATE_CMD =   "%s /sbin/lvcreate" % SUDO
LVREMOVE_CMD =   "%s /sbin/lvremove" % SUDO
MOUNT_CMD =      "%s /bin/mount" % SUDO
UMOUNT_CMD =     "%s /bin/umount" % SUDO
# There is a bug in this command with the locale, we set LANG=
XFS_FREEZE_CMD = "LANG= %s /usr/sbin/xfs_freeze" % SUDO

RSYNC_CMD = "%s /usr/bin/rsync" % SUDO

#----------------------------------------------------------------
# MySQL functions
def mysql_connect():
    dbconn = MySQLdb.connect (host = MYSQL_HOST,
                              port = MYSQL_PORT,
                              user = MYSQL_USER,
                              passwd = MYSQL_PASSWD,
                              db = MYSQL_DB)
    return dbconn

def mysql_lock_tables(dbconn):
    sqlcmd = "FLUSH TABLES WITH READ LOCK"

    print "Locking tables: %s" % sqlcmd

    cursor = dbconn.cursor()
    cursor.execute(sqlcmd)
    cursor.close()

def mysql_unlock_tables(dbconn):
    sqlcmd = "UNLOCK TABLES"

    print "Unlocking tables: %s" % sqlcmd

    cursor = dbconn.cursor()
    cursor.execute(sqlcmd)
    cursor.close()

#----------------------------------------------------------------
# LVM operations
class FailedLvmOperation(Exception):
    pass

# Get the fs type with a common shell script
def get_fs_type(fs_path):
    p = os.popen('mount | grep $(df %s |grep /dev |'\
                 'cut -f 1 -d " ") | cut -f 3,5 -d " "' % fs_path)
    (fs_mountpoint, fs_type) = p.readline().strip().split(' ')
    p.close()
    return (fs_mountpoint, fs_type)

def lvm_create_snapshot():
    # XFS filesystem supports freezing. That is convenient before the snapshot
    (fs_mountpoint, fs_type) = get_fs_type(DATA_FILES_PATH)
    if fs_type == 'xfs':
        print "Freezing '%s'" % fs_mountpoint
        os.system('%s -f %s' % (XFS_FREEZE_CMD, fs_mountpoint))

    newlv_name = "%s_backup_%ilv" % \
                    (DATA_FILES_LV.split('/')[-1], os.getpid())
    cmdline = "%s --snapshot %s -L%s --name %s" % \
        (LVCREATE_CMD, DATA_FILES_LV, SNAPSHOT_SIZE, newlv_name)

    print "Creating the snapshot backup LV '%s' from '%s'" % \
            (newlv_name, DATA_FILES_LV)
    print " # %s" % cmdline

    ret = os.system(cmdline)

    # Always unfreeze!!
    if fs_type == 'xfs':
        print "Unfreezing '%s'" % fs_mountpoint
        os.system('%s -u %s' % (XFS_FREEZE_CMD, fs_mountpoint))

    if ret != 0: raise FailedLvmOperation

    # Return the path to the device
    return '/'.join(DATA_FILES_LV.split('/')[:-1]+[newlv_name])

def lvm_remove_snapshot(lv_name):
    cmdline = "%s -f %s" % \
        (LVREMOVE_CMD, lv_name)

    print "Removing the snapshot backup LV '%s'" % lv_name
    print " # %s" % cmdline

    ret = os.system(cmdline)
    if ret != 0:
        raise FailedLvmOperation

#----------------------------------------------------------------
# Mount the filesystem
class FailedMountOperation(Exception):
    pass

def mount_snapshot(lv_name):
    # XFS filesystem needs nouuid option to mount snapshots
    (fs_mountpoint, fs_type) = get_fs_type(DATA_FILES_PATH)
    if fs_type == 'xfs':
        cmdline = "%s -o nouuid %s %s" % \
                    (MOUNT_CMD, lv_name, SNAPSHOT_MOUNTPOINT)
    else:
        cmdline = "%s %s %s" % (MOUNT_CMD, lv_name, SNAPSHOT_MOUNTPOINT)

    print "Mounting the snapshot backup LV '%s' on '%s'" % \
            (lv_name, SNAPSHOT_MOUNTPOINT)
    print " # %s" % cmdline

    ret = os.system(cmdline)
    if ret != 0:
        raise FailedMountOperation

def umount_snapshot(lv_name):
    cmdline = "%s %s" % (UMOUNT_CMD, SNAPSHOT_MOUNTPOINT)

    print "Unmounting the snapshot backup LV '%s' from '%s'" % \
            (lv_name, SNAPSHOT_MOUNTPOINT)
    print " # %s" % cmdline

    ret = os.system(cmdline)
    if ret != 0:
        raise FailedMountOperation

#----------------------------------------------------------------
# Perform the backup process. For instance, an rsync
class FailedBackupOperation(Exception):
    pass

def do_backup():
    cmdline = "%s --progress -av %s/ %s" % \
                (RSYNC_CMD, DATA_FILES_PATH, BACKUP_DESTINATION)

    print "Executing the backup"
    print " # %s" % cmdline

    ret = os.system(cmdline)
    if ret != 0:
        raise FailedBackupOperation

def main():
    dbconn = mysql_connect()
    mysql_lock_tables(dbconn)

    start_time = datetime.now()
    # Critical, tables are locked!
    snapshotlv = ''
    try:
        snapshotlv = lvm_create_snapshot()
    except:
        print "Backup failed."
        raise
    finally:
        mysql_unlock_tables(dbconn)
        dbconn.close()
        print "Tables had been locked for %s" % str(datetime.now()-start_time)

    try:
        mount_snapshot(snapshotlv)
        do_backup()
        umount_snapshot(snapshotlv)
        lvm_remove_snapshot(snapshotlv)
    except:
        print "Backup failed. Snapshot LV '%s' still exists. " % snapshotlv
        raise

    print "Backup completed. Elapsed time %s" % str(datetime.now()-start_time)

if __name__ == '__main__':
    main()

These is one of the proposed solutions for the job assessment commented in a previous post.

Provide a script manipulating the routing table to send all outgoing traffic originating from ipaddress: 85.14.228.248 through gw 85.14.228.254 and all other traffic through gateway 10.12.0.254

One basically have to execute these commands:

# Default route
ip route del default table 254
ip route add default via 192.168.1.1 dev wlan0 table 254

# alternative route and its rule
ip route del default table 1
ip route add default via 85.14.228.254 dev wlan0 table 1
ip rule del from 85.14.228.248
ip rule add from 85.14.228.248 table 1

I delete the previous default route and rule to ensure that the commands will be a success and will update the configuration.

A more convenient script could be:

#!/bin/bash

# Default route
DEFAULT_ROUTE=10.12.0.254
DEFAULT_DEV=eth0

# Create the diferent routes, where
#  NAME[id]  = name of routing table (for documentation purposes)
#  ROUTE[id] = destination
#  SRCS[id]  = list of routed ips
#  DEV[id]     = network device
#  id          = number of routing table (1..253)
#
NAME[1]=uplink1
ROUTE[1]="85.14.228.254"
DEV[1]=eth1
SRCS[1]="85.14.228.248"

#-----------------------------------------
# Set the "main" table
NAME[254]=main
ROUTE[254]=$DEFAULT_ROUTE
DEV[254]=$DEFAULT_DEV

# debug
#ip() { echo "> ip $*"; command ip $*; }

for i in {255..1}; do
[ ! -z "${ROUTE[$i]}" ] || continue

# Delete default route if exists
ip route list table $i | grep -q default && \
echo "Deleting default entry for route table ${NAME[$i]}($i)..." && \
ip route del default table $i

# Create the new table default route
echo "Creating route table '${NAME[$i]}($i)' with default via gw ${ROUTE[$i]}"
ip route add default via "${ROUTE[$i]}" dev ${DEV[$i]} table $i || continue

# Create
for ip in ${SRCS[i]}; do
# Delete rule if exists
ip rule list |grep -q "from $ip" && \
echo " - deleting rule from $ip..." && \
ip rule del from $ip

# Add the source rule
echo " + adding rule from $ip..."
ip rule add from $ip table $i
done
done

These is one of the proposed solutions for the job assessment commented in a previous post.

Using an Open Source solution, design a load balancer configuration that meets: redundancy, multiples subnets, and handle 500-1000Mbit of syn/ack/fin packets. Explain scalability of your design/configs.

The main problem that the load balancer design must solve in
web applications is the session stickiness (or persistence). The load balancer
design must be created according to the session replication policy of the architecture.
On the other hand, the load balancer must be designed to allow the upgrade and
maintenance of the servers.

Considering the architecture explained in Webserver architecture section, the stickiness restrictions are:

  • Session stickiness must be set for each site. Site failure means lose of all users sessions crated in that site.
  • Session stickiness must be set for each farm. Server failure is allowed (session will be recovered from the session DB backend)
  • Session stickiness for servers into the farm is optional (Could be set to take advantage of OS disk cache).

The software that I propose is HAProxy (http://haproxy.1wt.eu/):

The load balancer design consists of two layers:

  • One primary HAProxy LB, that will balance between sites.
    Configured with session stickiness to the site using an inserted cookie, SITEID.
    See primary-haproxy.conf.
  • One site LB in each site, balancing the site’s farms.
    Configured with session stickiness to the farm FARM_ID.
    See site1-haproxy.conf.

Extra comments:

  • Each layer can have several HAproxy instances, with the same configuration, configured with a failover solution or behind a Layer 4 load balancer. See http://haproxy.1wt.eu/download/1.3/doc/architecture.txt (Section 2) for examples.
  • Additionally a SSL frontend solution should be configured for SSL connections between the client
    and the primary load balancer. We can use plain HTTP between balancers and servers.
    I will not describe this element.
  • The solutions described in HAproxy architecture documentation, “4 Soft-stop for application maintenance”, can be used.
  • With HAProxy 1.4 you can dynamically control servers weight.
    A monitoring system can check the farms/servers health and tune the weight as needed.

This solution scales well. You simply need to add more servers, farms and sites.
Load-balancers can scale horizontally as commented.

Primary configuration:

#
# Primary Load Balancer configuration: primary-haproxy.conf
#
global
	log 127.0.0.1	local0
	log 127.0.0.1	local1 notice
	#log loghost	local0 info
	maxconn 40000
	user haproxy
	group haproxy
	daemon
	#debug
	#quiet

defaults
	log	global
	mode	http
	option	httplog
	option	dontlognull
	retries	3
	option redispatch
	maxconn	2000
	contimeout	5000
	clitimeout	50000
	srvtimeout	50000

listen primary_lb_1
    # We insert cookies, add headers => http mode
    mode http

    #------------------------------------
    # Bind to all address.
    #bind 0.0.0.0:10001
    # Bind to a clusterized virtual ip
    bind 192.168.10.1:10001 transparent

    #------------------------------------
    # Cookie persistence for PHP sessions. Options
    #  - rewrite PHPSESSID: will add the server label to the session id
    #cookie	PHPSESSID rewrite indirect
    #  - insert a cookie with the identifier.
    #    Use of postonly (session created in login form) or nocache to avoid be cached
    cookie SITEID insert postonly

    # We need to know the client ip in the end servers.
    # Inserts X-Forwarded-For. Needs httpclose (no Keep-Alive).
    option forwardfor
    option httpclose

    # Roundrobin is ok for HTTP requests.
    balance	roundrobin

    # The backend sites
    # Several options are possible:
    #  inter 2000 downinter 500 rise 2 fall 5 weight 100
    server site1 192.168.11.1:10001 cookie site1 check
    server site2 192.168.11.1:10002 cookie site2 check
    # etc..

Site configuration:

#
# Site 1 load balancer configuration: syte1-haproxy.conf
#
global
log 127.0.0.1    local0
log 127.0.0.1    local1 notice
#log loghost    local0 info
maxconn 40000
user haproxy
group haproxy
daemon
#debug
#quiet

defaults
log    global
mode    http
option    httplog
option    dontlognull
retries    3
option redispatch
maxconn    2000
contimeout    5000
clitimeout    50000
srvtimeout    50000

#------------------------------------
listen site1_lb_1
grace 20000 # don't kill us until 20 seconds have elapsed

# Bind to all address.
#bind 0.0.0.0:10001
# Bind to a clusterized virtual ip
bind 192.168.11.1:10001 transparent

# Persistence.
# The webservers in same farm share the session
# with memcached. The whole site has them in a DB backend.
mode http
cookie FARMID insert postonly

# Roundrobin is ok for HTTP requests.
balance roundrobin

# Farm 1 servers
server site1ws1 192.168.21.1:80 cookie farm1 check
server site1ws2 192.168.21.2:80 cookie farm1 check
# etc...

# Farm 2 servers
server site1ws17 192.168.21.17:80 cookie farm2 check
server site1ws18 192.168.21.18:80 cookie farm2 check
server site1ws19 192.168.21.19:80 cookie farm1 check
# etc..

These is one of the proposed solutions for the job assessment commented in a previous post.

Note that this was my reply it that moment, nowadays I would change my reply including automatic provisioning and automated configuration  based on puppet, chef… and other techniques. Also something about rollbacks, using LVM snapshots.

Question

Given a 500+ node webcluster in one costumer for one base code. Design a Gentoo/PHP version control. Propose solutions for OS upgrades of servers. Propose a plan and execution for PHP upgrades.  Please explain your choices.

Solution

About the OS/upgrades I will consider:

  • There are a limited number of hardware configurations. I will call them: hw-profile.
  • There is a preproduction environment, with servers of each hardware configuration.

In that case:

  • Each upgrade must be properly tested in the preproduction environment.

  • The preproduction servers will pre-compile the Gentoo packages
    for each hw-profile. Distributed compiling can be set.

  • There is a local Gentoo mirror and pre-compiled packages repository
    in the network, serving the binaries built for each hw-profile.

  • Each server will have associated its hw-profile repository and install the binaries:

    PORTAGE_BINHOST="ftp://gentoo-repository/$hw-profile"
    emerge --usepkg --getbinpkg <package>

The PHP upgrades can be distributed using rsync, in different location for each version,
and activated changing the apache/nginx configuration.

To plan the upgrades (both OS and PHP) I will consider the architecture
explained previously in Webserver architecture section, and the load balancing
solution described in Redundant load balancer design.

The upgrade requirements are:

  • HA, no lost of service due maintenance or upgrades.
  • Each request with an associated session must access to a
    webapp version equal or superior than previous request.
    This is important to ensure the application consistency
    (e.p. an user fills a form that is only available in the last version,
    session contains unexpected values…).

The upgrades can be divided in:

  • Non-disruptive OS upgrade: small OS software upgrades that are not related to
    the webservice (p.e. man, findutils, tar…). The upgrade can be performed online.

  • Disruptive OS upgrade: OS software that imply restart the service
    or the server (p.e. Apache, kernel, libc, ssl…):

    1. It will be upgraded only one member of each farms. First all members number 1,
      then number 2…
    2. The web service will be stopped during the upgrade.
      The other servers in the farm will serve without service disruption.

    This method provides homogeneous and little performance impact (100/16 = 6% servers down).

  • Application upgrade: Clients must access to equal or newer webapp versions:

    1. A complete farm must be stopped at the same time.
    2. The sessions sticked to this farm will be
      served by other farms in the same site (session stickiness to site).
      Session data will be recovered from DB backend.
    3. The memcached associated to this farm must be flushed.
    4. Once upgraded, all servers in the farm are started and can serve to new sessions.

    Load balancer stickiness ensure that the new sessions will access only to
    the upgraded farm. Except:

    • if all the servers of the farm fail at the same time after the upgrade.
    • if end user manipulates the cookies.

    In that case, control code can be added to the application to
    invalidate sessions from upper versions. Something like this:

    if ($app_version < $_SESSION['app_version'])
     session_destroy()
    elseif ($app_version != $_SESSION['app_version'])
     $_SESSION['app_version'] = $app_session

To perform the upgrades, cluster management tools can be used,
like MCollective (good if using puppet), func, fabric

 

If you are behind a proxy that allows HTTPS connections, you can use github via SSH without problems. To do so, you have to use the great tool connect.c ( http://bent.latency.net/bent/git/goto-san-connect-1.85/src/connect.html). As described in its homepage, this program tunnels a connection using a proxy, to allow SSH to connect to servers using a proxy.

You can configure connect as the ProxyCommand for ssh.github.com and github.com hosts in ~/.ssh/config. You can also set the Port to 443 aswell.

Basicly the process will be:

export PROXY=proxy:80

http_proxy=http://$PROXY wget http://www.taiyo.co.jp/~gotoh/ssh/connect.c -O /tmp/connect.c
gcc /tmp/connect.c -o ~/bin/connect 

cat >> ~/.ssh/config  <<EOF

Host ssh.github.com github.com
  Port 443
  HostName ssh.github.com
  IdentityFile $HOME/.ssh/id_rsa
  ProxyCommand $HOME/bin/connect -H proxy:80 %h %p

EOF

And ready!!

git clone git@github.com:keymon/facter.git facter

Easy, isn’t it?

Check connect.c documentation if you need to use an authenticated user in proxy.

How to define “hotkeys” in bash

Publicado: septiembre 20, 2010 en fast-tip, linux/unix, sysadmin, trick
Etiquetas:, , ,

For instance, I will define a hotkey to get manual page of current command without execute it (ideal for F1).

First, you get the code of the “hotkey” you want to use by pressing “Ctrl+U+<hotkey>”. For example:

* Ctrl+L: ^L
* Ctrl+J: ^J
* F1: ^[OP

This code may vary from terminal to terminal.

First you define an function, called single-man, to execute man of the first argument:

single-man() { man $1; }

Then, you add a line like this one in your .inputrc:

 "^[OP" "\C-A\C-K single-man \C-Y\C-M\C-Y"

What the hell does this? Well, when “F1” is pressed, in will simulate the press of “Ctrl+A”, that goes to the begining of the line, “Ctrl+K” that copies current line to clipboard, “Ctrl+Y” that pastes the clipboard, “Ctrl+M” that press Enter and Ctrl+Y that pastes the clipboard one more time.

I use this trick since several years ago.

Hoy he perdido algo de tiempo con una soberana tonteria: Creé un script para ejecutar en el /etc/cron.daily llamado “script.sh”… y nunca se ejecutaba. Revisé todo arriba y abajo, y finalmente ejecuté:

run-parts --report -v  --test /etc/cron.daily

y no salí el script… ¿que era? al final se me ocurrió quitarle el ‘.sh’ del final… y voilá.

Y la cosa no acabó ahí. Otra definición en /etc/cron.d/zabbix.cron no cargaba… el mismo problema, sobraba el ‘.’.

En fins…

In a previous post I commented the problems related to LDAP integration of Linux with LDAP. I proposed several solutions and commented that a good configuration can be enough. Tuning the configuration, trying to avoid buggy code, minimizing locks and delays, etc…

In this post I will comment a configuration that is quite stable at the moment, using Suse 11 SP2 with Active Directory 2003 + SFU.

(más…)

I have been asked to install AccessStream application on linux to test it.

But this application has not ANY documentation or install procedure. At the end I managed to install it, here is the procedure.

(más…)

Un amigo mío me comenta que necesitaba ampliar o incrementar el disco de un hosting en internet, y me preguntaba cual sería la mejor forma. Obviamente hay que minimizar el tiempo de caída.

Allí, lo único que le hicieron fué ampliar el disco iSCSI asignado en 50GB. El disco está particionado en 2 particiones, una de boot y otra de datos, y quiere ampliar la de datos (ficheros en fs en ext3) con la mínima disrupción. No usa LVM.

La empresa de hosting le propone reiniciar en modo “administración” (con una imagen en red) y borrar y crear de nuevo la partición, para luego redimensionar el fs con resize2fs… Pero no se lo recomiendo porque:

  • Es una perdida de servicio muy grande.
  • Cada vez que amplie tendrá que reiniciar.
  • no me fio de resize2fs, ya me falló con anterioridad.

Yo le propongo que se pase a LVM + xfs. E incluso puede hacerlo sin reiniciar el servidor, en caliente, y con una parada de servicio mínima (<1min). En este post comento el procedimiento con comandos simples y disponibles en practiamente todas las distribuciones.

El proceso seria:

  1. Hacer backup. Siempre.
  2. Reescanear buses y discos scsi para detectar nuevo tamaño de disco.
  3. Reparticionar para crear una nueva partición con el nuevo espacio. Es mejor extendida, para poder ampliar en futuras ocasiones.
  4. Configurar una LV con LVM en el nuevo espacio (pvcreate, vgcreate, lvcreate).
  5. Montar y clonar los datos con rsync.
  6. Parar el servicio, resincronizar los últimos cambios con rsync, intercambiar el punto de montaje y arrancar el servicio.

En el paso 2 nos encontramos un problema. Linux, al ser el mismo de boot y estar montado no va a recargar la tabla al salir del fdisk. Pero por lo visto el comando partprobe, que viene con parted, es capaz de crear las nuevas particiones aún usando ese disco :).

Así que simplemente los pasos son:

  1. Decirle al hosting que incremente el disco.
  2. Reescanear las fibras con este sencillo script:
    cat > reescanea-scsi <<"EOF"
    #!/bin/bash
    
    for fn in /sys/class/scsi_host/*
    do
            host=$(basename $fn)
            echo "Scanning $host ... "
            if [ -d $fn ]; then
                    echo "- - -" > /sys/class/scsi_host/$host/scan
            else
                    echo "ERROR, device not found : '$fn'"
            fi
    done
    
    for disk in /sys/class/scsi_device/*/device/rescan; do
            echo "Rescanning device $disk ..."
            echo 1 > "$disk"
    done
    
    exit 0
    EOF
    chmod +x reescanea-scsi
    ./reescanea-scsi
    

    La salida será algo así. Vemos que nos cambia el tamaño:

    # ./reescanea-scsi
    Scanning host0 ...
    Scanning host1 ...
    Scanning host2 ...
    Rescanning device /sys/class/scsi_device/0:0:0:0/device/rescan ...
    Rescanning device /sys/class/scsi_device/0:0:2:0/device/rescan ...
    Rescanning device /sys/class/scsi_device/0:0:3:0/device/rescan ...
    Rescanning device /sys/class/scsi_device/1:0:0:0/device/rescan ...
    # dmesg|grep sda 
    sd 0:0:0:0: [sda] 20971520 512-byte hardware sectors: (10.7GB/10.0GiB)
    sd 0:0:0:0: [sda] Test WP failed, assume Write Enabled
    sd 0:0:0:0: [sda] Cache data unavailable
    sd 0:0:0:0: [sda] Assuming drive cache: write through
    sd 0:0:0:0: [sda] 20971520 512-byte hardware sectors: (10.7GB/10.0GiB)
    sd 0:0:0:0: [sda] Test WP failed, assume Write Enabled
    sd 0:0:0:0: [sda] Cache data unavailable
    sd 0:0:0:0: [sda] Assuming drive cache: write through
     sda: sda1 sda2 sda3
    sd 0:0:0:0: [sda] Attached SCSI disk
    Adding 1052248k swap on /dev/sda2.  Priority:1 extents:1 across:1052248k
    EXT3 FS on sda1, internal journal
    sd 0:0:0:0: [sda] 23068672 512-byte hardware sectors: (11.8GB/11.0GiB)
    sd 0:0:0:0: [sda] Write Protect is off
    sd 0:0:0:0: [sda] Mode Sense: 03 00 00 00
    sd 0:0:0:0: [sda] Cache data unavailable
    sd 0:0:0:0: [sda] Assuming drive cache: write through
    sda: detected capacity change from 10737418240 to 11811160064
    
  3. Creamos la partición extendida con fdisk (o otro similar):
    # fdisk /dev/sda
    
    The number of cylinders for this disk is set to 1435.
    There is nothing wrong with that, but this is larger than 1024,
    and could in certain setups cause problems with:
    1) software that runs at boot time (e.g., old versions of LILO)
    2) booting and partitioning software from other OSs
       (e.g., DOS FDISK, OS/2 FDISK)
    
    Command (m for help): p
    
    Disk /dev/sda: 11.8 GB, 11811160064 bytes
    255 heads, 63 sectors/track, 1435 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes
    Disk identifier: 0x000a4c74
    
       Device Boot      Start         End      Blocks   Id  System
    /dev/sda1   *           1          13      104391   83  Linux
    /dev/sda2              14         144     1052257+  82  Linux swap / Solaris
    /dev/sda3             145        1305     9325732+  8e  Linux LVM
    
    Command (m for help): n
    Command action
       e   extended
       p   primary partition (1-4)
    e
    Selected partition 4
    First cylinder (1306-1435, default 1306):
    Using default value 1306
    Last cylinder, +cylinders or +size{K,M,G} (1306-1435, default 1435):
    Using default value 1435
    
    Command (m for help): p
    
    Disk /dev/sda: 11.8 GB, 11811160064 bytes
    255 heads, 63 sectors/track, 1435 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes
    Disk identifier: 0x000a4c74
    
       Device Boot      Start         End      Blocks   Id  System
    /dev/sda1   *           1          13      104391   83  Linux
    /dev/sda2              14         144     1052257+  82  Linux swap / Solaris
    /dev/sda3             145        1305     9325732+  8e  Linux LVM
    /dev/sda4            1306        1435     1044225    5  Extended
    
    Command (m for help): n
    First cylinder (1306-1435, default 1306):
    Using default value 1306
    Last cylinder, +cylinders or +size{K,M,G} (1306-1435, default 1435):
    Using default value 1435
    
    Command (m for help): p
    
    Disk /dev/sda: 11.8 GB, 11811160064 bytes
    255 heads, 63 sectors/track, 1435 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes
    Disk identifier: 0x000a4c74
    
       Device Boot      Start         End      Blocks   Id  System
    /dev/sda1   *           1          13      104391   83  Linux
    /dev/sda2              14         144     1052257+  82  Linux swap / Solaris
    /dev/sda3             145        1305     9325732+  8e  Linux LVM
    /dev/sda4            1306        1435     1044225    5  Extended
    /dev/sda5            1306        1435     1044193+  83  Linux
    
    Command (m for help): w
    The partition table has been altered!
    
    Calling ioctl() to re-read partition table.
    
    WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
    The kernel still uses the old table.
    The new table will be used at the next reboot.
    Syncing disks.
    
    dcsrvmonits1:/home/invitado # ls /dev/sda5
    ls: cannot access /dev/sda5: No such file or directory
    

    Observamos cómo falla la ioctl de recarga de particiones y no detecta la nueva partición… Probamos con partprobe:

    dcsrvmonits1:/home/invitado # partprobe /dev/sda
    dcsrvmonits1:/home/invitado # ls /dev/sda5
    /dev/sda5
    

    Así funciona. Cosa curiosa, no sale ningún mensaje en dmesg.

  4. Configuramos LVM… mirate el manual para saber más:
    # pvcreate /dev/sda5
    File descriptor 5 left open
      No physical volume label read from /dev/sda5
      Physical volume "/dev/sda5" successfully created
    # pvdisplay
    File descriptor 5 left open
      "/dev/sda5" is a new physical volume of "1019.72 MB"
      --- NEW Physical volume ---
      PV Name               /dev/sda5
      VG Name
      PV Size               1019.72 MB
      Allocatable           NO
      PE Size (KByte)       0
      Total PE              0
      Free PE               0
      Allocated PE          0
      PV UUID               LpvfCq-gzrR-tjC5-N3E2-dA6x-Hmoi-UlPzBK
    
    # vgcreate datavg /dev/sda5
    File descriptor 5 left open
      Volume group "datavg" successfully created
    # vgdisplay datavg
    File descriptor 5 left open
      --- Volume group ---
      VG Name               datavg
      System ID
      Format                lvm2
      Metadata Areas        1
      Metadata Sequence No  1
      VG Access             read/write
      VG Status             resizable
      MAX LV                0
      Cur LV                0
      Open LV               0
      Max PV                0
      Cur PV                1
      Act PV                1
      VG Size               1016.00 MB
      PE Size               4.00 MB
      Total PE              254
      Alloc PE / Size       0 / 0
      Free  PE / Size       254 / 1016.00 MB
      VG UUID               1wC2Vb-omIq-zpDJ-pnUg-oU2f-HaXP-sp29XD
    
    # lvcreate -n reposlv datavg -L 1016.00M
    File descriptor 5 left open
      Logical volume "reposlv" created
    # lvdisplay
      --- Logical volume ---
      LV Name                /dev/datavg/reposlv
      VG Name                datavg
      LV UUID                bIrslV-vSlB-elpP-no2v-B1yt-FO2G-CMjq9l
      LV Write Access        read/write
      LV Status              available
      # open                 0
      LV Size                1016.00 MB
      Current LE             254
      Segments               1
      Allocation             inherit
      Read ahead sectors     auto
      - currently set to     256
      Block device           253:7
    
  1. Listo, formateamos, montamos y sincronizamos:
    # mkfs.xfs /dev/datavg/reposlv
    meta-data=/dev/datavg/reposlv    isize=256    agcount=4, agsize=65024 blks
             =                       sectsz=512   attr=2
    data     =                       bsize=4096   blocks=260096, imaxpct=25
             =                       sunit=0      swidth=0 blks
    naming   =version 2              bsize=4096   ascii-ci=0
    log      =internal log           bsize=4096   blocks=1200, version=2
             =                       sectsz=512   sunit=0 blks, lazy-count=0
    realtime =none                   extsz=4096   blocks=0, rtextents=0
    # mkdir /mnt/repos.new
    # mount /dev/datavg/reposlv /mnt/repos.new
    

    Clonamos:

    # rsync -av --delete /mnt/repos/ /mnt/repos.new
    
  2. Paramos un segundo el servicio, volvemos a sincronizar, damos el cambiazo y arrancamos el servicio. Se puede hacer en un script de una tacada:
    apachectl stop
    rsync -av --delete /mnt/repos/ /mnt/repos.new
    umount /mnt/repos
    umount /mnt/repos.new
    mount /dev/datavg/reposlv /mnt/repos
    apachectl start
    # Actualiza el fstab
    sed -i 's|/dev/sda2|/dev/datavg/reposlv|' /etc/fstab
    
  3. Por último, despues de comprobar que todo está ok, agregamos el viejo espacio al VG y aumentamos así el LV. En caliente :):
    pvcreate /dev/sda2
    vgextend datavg /dev/sda2
    lvextend -l FREE /dev/datavg/reposlv
    xfs_growfs /dev/datavg/reposlv
    

Simple ¿no?