Posts etiquetados ‘linux’

If you are behind a proxy that allows HTTPS connections, you can use github via SSH without problems. To do so, you have to use the great tool connect.c ( http://bent.latency.net/bent/git/goto-san-connect-1.85/src/connect.html). As described in its homepage, this program tunnels a connection using a proxy, to allow SSH to connect to servers using a proxy.

You can configure connect as the ProxyCommand for ssh.github.com and github.com hosts in ~/.ssh/config. You can also set the Port to 443 aswell.

Basicly the process will be:

export PROXY=proxy:80

http_proxy=http://$PROXY wget http://www.taiyo.co.jp/~gotoh/ssh/connect.c -O /tmp/connect.c
gcc /tmp/connect.c -o ~/bin/connect 

cat >> ~/.ssh/config  <<EOF

Host ssh.github.com github.com
  Port 443
  HostName ssh.github.com
  IdentityFile $HOME/.ssh/id_rsa
  ProxyCommand $HOME/bin/connect -H proxy:80 %h %p

EOF

And ready!!

git clone git@github.com:keymon/facter.git facter

Easy, isn’t it?

Check connect.c documentation if you need to use an authenticated user in proxy.

How to define “hotkeys” in bash

Publicado: septiembre 20, 2010 en fast-tip, linux/unix, sysadmin, trick
Etiquetas:, , ,

For instance, I will define a hotkey to get manual page of current command without execute it (ideal for F1).

First, you get the code of the “hotkey” you want to use by pressing “Ctrl+U+<hotkey>”. For example:

* Ctrl+L: ^L
* Ctrl+J: ^J
* F1: ^[OP

This code may vary from terminal to terminal.

First you define an function, called single-man, to execute man of the first argument:

single-man() { man $1; }

Then, you add a line like this one in your .inputrc:

 "^[OP" "\C-A\C-K single-man \C-Y\C-M\C-Y"

What the hell does this? Well, when “F1” is pressed, in will simulate the press of “Ctrl+A”, that goes to the begining of the line, “Ctrl+K” that copies current line to clipboard, “Ctrl+Y” that pastes the clipboard, “Ctrl+M” that press Enter and Ctrl+Y that pastes the clipboard one more time.

I use this trick since several years ago.

I will describe my configuration of puppetmaster server and puppet dashboard server running into the Debian’s Apache installation, but:

  • Using a custom or different user, not www-data and root. This is good to keep all puppet configuration and data with a different user than www-data and root.
  • Using a custom configuration directory, not default apache directory: /etc/apache2
  • You can issolate puppet server from the rest of apache applications.

First I recommend you to read the official documentation:

It is supposed that you have a running puppet installation.

(más…)

In a previous post I commented the problems related to LDAP integration of Linux with LDAP. I proposed several solutions and commented that a good configuration can be enough. Tuning the configuration, trying to avoid buggy code, minimizing locks and delays, etc…

In this post I will comment a configuration that is quite stable at the moment, using Suse 11 SP2 with Active Directory 2003 + SFU.

(más…)

I have been asked to install AccessStream application on linux to test it.

But this application has not ANY documentation or install procedure. At the end I managed to install it, here is the procedure.

(más…)

To integrate a Linux system with a centralized user directory (like Microsoft Active Directory) the usual solution is to configure Kerberos for Authentication (password/credential checking) and LDAP for Authorization and Access Control. The “standarized” way to implement this is using libpam_krb5libnss_ldap (by padl software) and nscd (from libc).

Kerberos integration works pretty well and I do not have too many issues with it, but I can not say the same from libnss_ldap and nscd.

In this post I will explain the anoying problems that you can find using libnss_ldap and nscd, and propose some solutions and configurations that will make it work properly. I also recomend read a previous post about the problems and solutions with connecting an Unix server to Active directory (Spanish post).

Read this article if you are experiencing problems with nscd+libnss_ldap (quoting http://www.nico.schottelius.org/blog/nscd-bugs/):

  • Sometimes it consumes 100% cpu (and does not stop that until being killed)
  • Sometimes it just crashes.
  • Sometimes it causes users to “vanish”
  • Sometimes it hangs and thus slows down the whole system
  • Sometimes it makes all the host work slow
  • Sometimes login a host or execute sudo/su takes a lot of time or never logins.
  • Sometimes sudo or su dies with “Segmentation Fault”
  • Sometimes a simple ‘ls’ command takes minutes.
  • etc…

(más…)

Puppet architecture needs a client to connect to the server to load the configuration usin a pull schema. But I do not like to have more and more daemons around and some people suggest avoid that , so I decided to execute puppet using ‘–onetime’ option from cron.

Obviously, I want to configure this using puppet itself. And we must ensure that the clients are executed at different times, not all at the same minute.

I searched the net and I found several aproaches to do this. There are also feature requests.

I read somewhere that the new function fqdn_rand() could be used, as proposed in the feature request and posted in this mail from Brice Figureau. I can not find where the hell the snippet was. At the end, I found this pastie by Jhon Goebel.

I will post my version here just to keep it wrote down.

 $first = fqdn_rand(30)
 $second = fqdn_rand(30) + 30
 cron { "cron.puppet.onetime":
 command => "/srv/scripts/puppet/puppet.ctl.sh onetime > /dev/null",
 user => "root",
 minute => [ $first, $second ],
 require => File["/srv/scripts/puppet/puppet.ctl.sh"],
 }

… this is another random thinking from keymon (https://keymon.wordpress.com)


									

Un amigo mío me comenta que necesitaba ampliar o incrementar el disco de un hosting en internet, y me preguntaba cual sería la mejor forma. Obviamente hay que minimizar el tiempo de caída.

Allí, lo único que le hicieron fué ampliar el disco iSCSI asignado en 50GB. El disco está particionado en 2 particiones, una de boot y otra de datos, y quiere ampliar la de datos (ficheros en fs en ext3) con la mínima disrupción. No usa LVM.

La empresa de hosting le propone reiniciar en modo “administración” (con una imagen en red) y borrar y crear de nuevo la partición, para luego redimensionar el fs con resize2fs… Pero no se lo recomiendo porque:

  • Es una perdida de servicio muy grande.
  • Cada vez que amplie tendrá que reiniciar.
  • no me fio de resize2fs, ya me falló con anterioridad.

Yo le propongo que se pase a LVM + xfs. E incluso puede hacerlo sin reiniciar el servidor, en caliente, y con una parada de servicio mínima (<1min). En este post comento el procedimiento con comandos simples y disponibles en practiamente todas las distribuciones.

El proceso seria:

  1. Hacer backup. Siempre.
  2. Reescanear buses y discos scsi para detectar nuevo tamaño de disco.
  3. Reparticionar para crear una nueva partición con el nuevo espacio. Es mejor extendida, para poder ampliar en futuras ocasiones.
  4. Configurar una LV con LVM en el nuevo espacio (pvcreate, vgcreate, lvcreate).
  5. Montar y clonar los datos con rsync.
  6. Parar el servicio, resincronizar los últimos cambios con rsync, intercambiar el punto de montaje y arrancar el servicio.

En el paso 2 nos encontramos un problema. Linux, al ser el mismo de boot y estar montado no va a recargar la tabla al salir del fdisk. Pero por lo visto el comando partprobe, que viene con parted, es capaz de crear las nuevas particiones aún usando ese disco :).

Así que simplemente los pasos son:

  1. Decirle al hosting que incremente el disco.
  2. Reescanear las fibras con este sencillo script:
    cat > reescanea-scsi <<"EOF"
    #!/bin/bash
    
    for fn in /sys/class/scsi_host/*
    do
            host=$(basename $fn)
            echo "Scanning $host ... "
            if [ -d $fn ]; then
                    echo "- - -" > /sys/class/scsi_host/$host/scan
            else
                    echo "ERROR, device not found : '$fn'"
            fi
    done
    
    for disk in /sys/class/scsi_device/*/device/rescan; do
            echo "Rescanning device $disk ..."
            echo 1 > "$disk"
    done
    
    exit 0
    EOF
    chmod +x reescanea-scsi
    ./reescanea-scsi
    

    La salida será algo así. Vemos que nos cambia el tamaño:

    # ./reescanea-scsi
    Scanning host0 ...
    Scanning host1 ...
    Scanning host2 ...
    Rescanning device /sys/class/scsi_device/0:0:0:0/device/rescan ...
    Rescanning device /sys/class/scsi_device/0:0:2:0/device/rescan ...
    Rescanning device /sys/class/scsi_device/0:0:3:0/device/rescan ...
    Rescanning device /sys/class/scsi_device/1:0:0:0/device/rescan ...
    # dmesg|grep sda 
    sd 0:0:0:0: [sda] 20971520 512-byte hardware sectors: (10.7GB/10.0GiB)
    sd 0:0:0:0: [sda] Test WP failed, assume Write Enabled
    sd 0:0:0:0: [sda] Cache data unavailable
    sd 0:0:0:0: [sda] Assuming drive cache: write through
    sd 0:0:0:0: [sda] 20971520 512-byte hardware sectors: (10.7GB/10.0GiB)
    sd 0:0:0:0: [sda] Test WP failed, assume Write Enabled
    sd 0:0:0:0: [sda] Cache data unavailable
    sd 0:0:0:0: [sda] Assuming drive cache: write through
     sda: sda1 sda2 sda3
    sd 0:0:0:0: [sda] Attached SCSI disk
    Adding 1052248k swap on /dev/sda2.  Priority:1 extents:1 across:1052248k
    EXT3 FS on sda1, internal journal
    sd 0:0:0:0: [sda] 23068672 512-byte hardware sectors: (11.8GB/11.0GiB)
    sd 0:0:0:0: [sda] Write Protect is off
    sd 0:0:0:0: [sda] Mode Sense: 03 00 00 00
    sd 0:0:0:0: [sda] Cache data unavailable
    sd 0:0:0:0: [sda] Assuming drive cache: write through
    sda: detected capacity change from 10737418240 to 11811160064
    
  3. Creamos la partición extendida con fdisk (o otro similar):
    # fdisk /dev/sda
    
    The number of cylinders for this disk is set to 1435.
    There is nothing wrong with that, but this is larger than 1024,
    and could in certain setups cause problems with:
    1) software that runs at boot time (e.g., old versions of LILO)
    2) booting and partitioning software from other OSs
       (e.g., DOS FDISK, OS/2 FDISK)
    
    Command (m for help): p
    
    Disk /dev/sda: 11.8 GB, 11811160064 bytes
    255 heads, 63 sectors/track, 1435 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes
    Disk identifier: 0x000a4c74
    
       Device Boot      Start         End      Blocks   Id  System
    /dev/sda1   *           1          13      104391   83  Linux
    /dev/sda2              14         144     1052257+  82  Linux swap / Solaris
    /dev/sda3             145        1305     9325732+  8e  Linux LVM
    
    Command (m for help): n
    Command action
       e   extended
       p   primary partition (1-4)
    e
    Selected partition 4
    First cylinder (1306-1435, default 1306):
    Using default value 1306
    Last cylinder, +cylinders or +size{K,M,G} (1306-1435, default 1435):
    Using default value 1435
    
    Command (m for help): p
    
    Disk /dev/sda: 11.8 GB, 11811160064 bytes
    255 heads, 63 sectors/track, 1435 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes
    Disk identifier: 0x000a4c74
    
       Device Boot      Start         End      Blocks   Id  System
    /dev/sda1   *           1          13      104391   83  Linux
    /dev/sda2              14         144     1052257+  82  Linux swap / Solaris
    /dev/sda3             145        1305     9325732+  8e  Linux LVM
    /dev/sda4            1306        1435     1044225    5  Extended
    
    Command (m for help): n
    First cylinder (1306-1435, default 1306):
    Using default value 1306
    Last cylinder, +cylinders or +size{K,M,G} (1306-1435, default 1435):
    Using default value 1435
    
    Command (m for help): p
    
    Disk /dev/sda: 11.8 GB, 11811160064 bytes
    255 heads, 63 sectors/track, 1435 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes
    Disk identifier: 0x000a4c74
    
       Device Boot      Start         End      Blocks   Id  System
    /dev/sda1   *           1          13      104391   83  Linux
    /dev/sda2              14         144     1052257+  82  Linux swap / Solaris
    /dev/sda3             145        1305     9325732+  8e  Linux LVM
    /dev/sda4            1306        1435     1044225    5  Extended
    /dev/sda5            1306        1435     1044193+  83  Linux
    
    Command (m for help): w
    The partition table has been altered!
    
    Calling ioctl() to re-read partition table.
    
    WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
    The kernel still uses the old table.
    The new table will be used at the next reboot.
    Syncing disks.
    
    dcsrvmonits1:/home/invitado # ls /dev/sda5
    ls: cannot access /dev/sda5: No such file or directory
    

    Observamos cómo falla la ioctl de recarga de particiones y no detecta la nueva partición… Probamos con partprobe:

    dcsrvmonits1:/home/invitado # partprobe /dev/sda
    dcsrvmonits1:/home/invitado # ls /dev/sda5
    /dev/sda5
    

    Así funciona. Cosa curiosa, no sale ningún mensaje en dmesg.

  4. Configuramos LVM… mirate el manual para saber más:
    # pvcreate /dev/sda5
    File descriptor 5 left open
      No physical volume label read from /dev/sda5
      Physical volume "/dev/sda5" successfully created
    # pvdisplay
    File descriptor 5 left open
      "/dev/sda5" is a new physical volume of "1019.72 MB"
      --- NEW Physical volume ---
      PV Name               /dev/sda5
      VG Name
      PV Size               1019.72 MB
      Allocatable           NO
      PE Size (KByte)       0
      Total PE              0
      Free PE               0
      Allocated PE          0
      PV UUID               LpvfCq-gzrR-tjC5-N3E2-dA6x-Hmoi-UlPzBK
    
    # vgcreate datavg /dev/sda5
    File descriptor 5 left open
      Volume group "datavg" successfully created
    # vgdisplay datavg
    File descriptor 5 left open
      --- Volume group ---
      VG Name               datavg
      System ID
      Format                lvm2
      Metadata Areas        1
      Metadata Sequence No  1
      VG Access             read/write
      VG Status             resizable
      MAX LV                0
      Cur LV                0
      Open LV               0
      Max PV                0
      Cur PV                1
      Act PV                1
      VG Size               1016.00 MB
      PE Size               4.00 MB
      Total PE              254
      Alloc PE / Size       0 / 0
      Free  PE / Size       254 / 1016.00 MB
      VG UUID               1wC2Vb-omIq-zpDJ-pnUg-oU2f-HaXP-sp29XD
    
    # lvcreate -n reposlv datavg -L 1016.00M
    File descriptor 5 left open
      Logical volume "reposlv" created
    # lvdisplay
      --- Logical volume ---
      LV Name                /dev/datavg/reposlv
      VG Name                datavg
      LV UUID                bIrslV-vSlB-elpP-no2v-B1yt-FO2G-CMjq9l
      LV Write Access        read/write
      LV Status              available
      # open                 0
      LV Size                1016.00 MB
      Current LE             254
      Segments               1
      Allocation             inherit
      Read ahead sectors     auto
      - currently set to     256
      Block device           253:7
    
  1. Listo, formateamos, montamos y sincronizamos:
    # mkfs.xfs /dev/datavg/reposlv
    meta-data=/dev/datavg/reposlv    isize=256    agcount=4, agsize=65024 blks
             =                       sectsz=512   attr=2
    data     =                       bsize=4096   blocks=260096, imaxpct=25
             =                       sunit=0      swidth=0 blks
    naming   =version 2              bsize=4096   ascii-ci=0
    log      =internal log           bsize=4096   blocks=1200, version=2
             =                       sectsz=512   sunit=0 blks, lazy-count=0
    realtime =none                   extsz=4096   blocks=0, rtextents=0
    # mkdir /mnt/repos.new
    # mount /dev/datavg/reposlv /mnt/repos.new
    

    Clonamos:

    # rsync -av --delete /mnt/repos/ /mnt/repos.new
    
  2. Paramos un segundo el servicio, volvemos a sincronizar, damos el cambiazo y arrancamos el servicio. Se puede hacer en un script de una tacada:
    apachectl stop
    rsync -av --delete /mnt/repos/ /mnt/repos.new
    umount /mnt/repos
    umount /mnt/repos.new
    mount /dev/datavg/reposlv /mnt/repos
    apachectl start
    # Actualiza el fstab
    sed -i 's|/dev/sda2|/dev/datavg/reposlv|' /etc/fstab
    
  3. Por último, despues de comprobar que todo está ok, agregamos el viejo espacio al VG y aumentamos así el LV. En caliente :):
    pvcreate /dev/sda2
    vgextend datavg /dev/sda2
    lvextend -l FREE /dev/datavg/reposlv
    xfs_growfs /dev/datavg/reposlv
    

Simple ¿no?

I will briefly describe how to set a cups-pdf on cups on Linux and configure AIX to use it. It is an easy task.

  1. Install on Linux cups and cups-pdf (for SucksE (Suse) you can find it in openSuse repositories).

    The cups-pdf package configures automaticly a printer called “cups-pdf”

  2. You can access the CUPS configuration page via http://localhost:631. If it is in a remote server, you can forward the port via SSH: “ssh -R 6310:localhost:631 host” and access via http://localhost:631.
  3. To use it on AIX, you need to configure the LPD protocol enabling cups-lpd in xinetd: On suse you must enable it in /etc/xinetd.d/cups-lpd.NOTE: You must disable the usage of banners (added by default by cups-lpd when converting from lpd to ipp) or you will get always a file called “Test_Page.pdf” with only the banner. I think that newer versions of cups solve this problem. To do that, you must add to cups-lpd the option -o job-sheets=none
sed 's/\(disable.*=\).*/\1 no/' -i /etc/xinetd.d/cups-lpd	
grep -q job-sheets=none /etc/xinetd.d/cups-lpd || sed 's/\(server_args.*=.*\)/\1 -o job-sheets=none/' -i /etc/xinetd.d/cups-lpd
/etc/init.d/xinetd reload

Finally on AIX, you can create you new printer as a BSD printer:

/usr/lib/lpd/pio/etc/piomisc_ext mkpq_remote_ext  -q 'cups-pdf' -h 'remoteserver' -r 'cups-pdf' -t 'bsd' -C 'FALSE' -d 'Virtual PDF printer on remoteserver'

That is all. You can use your virtual pdf printer on AIX: ls | lp -d cups-pdf

You may want tune some cups-pdf settings in /etc/cups/cups-pdf.conf, like:

  • UserUMask 0007: This option affects the “umask” default ACL configuration. If you set 0077 it will set umask=— in final PDF, I do not known why :?
    ### Key: UserUMask
    ##  umask for user output of known users
    ##  changing this can introduce security leaks if confidential
    ##  information is processed!
    ### Default: 0077
    
    UserUMask 0007
    
  • Label 1, to avoid overwrites…
    ### Key: Label
    ##  label all jobs with a unique job-id in order to avoid overwriting old
    ##  files in case new ones with identical names are created; always true for
    ##  untitled documents
    ##  0: label untitled documents only, 1: label all documents
    ### Default: 0
    
  • Paths, etc….

BSoD for linux/unix/console

Publicado: junio 7, 2010 en fast-tip, humor, Personal, script
Etiquetas:, , ,

I do not remember where I got this, but it is fun. I will put it in the motd of my hosts on April Fools’ day.

This script will display on a console a windows like BSoD (Blue Script of Day):

                                    Linux ws         An exception 0E has occurred at 0028:C0018DBA in VxD IFSMGR(01) +        0000340A.  This was called from 0028:C0034118 in VxD NDIS(01) +        00000D7C.  It may be possible to continue normally.         *  Press any key to attempt to continue        *  Press CTRL+ALT+DEL to restart your computer. You will           lose any unsaved information in all applications                             Press Any key to continue. (más…)