path: root/080_blog/00013_Old-Software-Fools-Mediawiki-Content/index.md

Fool’s Wiki Knowledge
=====================

    February 24, 2016

Migrated from our former Mediawiki:

Welcome to the Foolo-Pedia! Foolo-Pedia is a growing collection of interesting, important, ridiculous or useless information gathered and refined by the SoftwareFools team for all Friends of Fools. We decided to mainly keep this Wiki in english to be compatible with all our costumers and friends. Some destinct pages maybe in polish, german or klingon.

Articles which might be of universal interest, divided by area and topic.

## Minilinux

    software fools minimal linux

    1. usb stick
    2. create one big partition (2gb)
    3. mkefs.ext4 /dev/sdx1
    4. mount /dev/sdx1 /mnt
    5. compile kernel
    6. cp bzImage /mnt
    7. boot

    8. 
    grub> search /bzImage
    grub> linux /bzImage root=??

    PROBLEMS WITH kernel config!

## Kernel Building

    cd linux-source-[xxx]
    make mrproper
    cp someconfig .config -i
    make oldconfig
    make menuconfig
    make localmodconfig
    make localyesconfig
    make
    (OPT) make modules
    su
    make install
    (OPT) make modules_install
    (OPT) update-intiramfs -c -k [kernel-postfix]

    (REMOVE UNWANTED FILES FROM BOOT!)
    update-grub
    grub-install /dev/sda
    reboot

## GDB

    http://www.cprogramming.com/debuggers.html

    use gcc or clang with : -ggdb -O0 

    h - help 

    r - run
    c - continue
    n - next
    s - step

    b - break <line>
    p - print <var>

    wa - watch <var>

    bt
    frame <number>
    list
    info locals

    set <var> = <value>
    set substitute-path FROM TO

    call <func>

## clang

    http://llvm.org/releases/3.7.0/docs/CMake.html
    http://clang.llvm.org/get_started.html
    put MAKEFLAGS="-j8" or similar in front of your CMake invocations.

    extracted llvm to ~/temp/clang/llvm-3.7.0.src/
    extracted clang to ~/temp/clang/llvm-3.7.0.src/tools/clang/ 
    extracted compiler-rt to ~/temp/clang/llvm-3.7.0.src/projects/compiler-rt 
    extracted libcxx to ~/temp/clang/llvm-3.7.0.src/projects/libcxx

    $ cd ~/temp/clang/llvm-3.7.0-build
    $ cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/home/miguel/opt/llvm-3.7.0 ../llvm-3.7.0.src
    $ cmake --build .
    $ cmake --build . --target install

## XMPP

### Idea

Today I figured out, once again, that there are many open-source and free (as in freedom) alternatives to all those chat clients and their proprietary protocols, which we dislike and want to avoid. Taking it seriously I decided to set up a XMPP Server on our Softwarefools Server (http://www.softwarefools.com) to finally enable secure and comfortable communication between me, myself and my fellow peers.

### Starting Point

* We have a Debian Virtual Private Server (VPS) Running wheezy (stable)
* I just found that tutorial: http://wiki.linuxwall.info/doku.php/en:ressources:articles:ejabberd

### I. get package

    # apt-get install ejabberd

### II. adapt config

In /etc/ejabberd/ejabberd.cfg adapt the hosts line as that: (if you wonder this is Erlang)

    %% Hostname
    {hosts, ["softwarefools.com"]}.

### III. setup certificate

    # mkdir /etc/ejabberd/certs

For now we skip the rest of that step and proceed with the self-signed certificate provided by Debian. (/etc/ejabberd/ejabberd.pem)

### IV. restart server

    # /etc/init.d/ejabberd restart

### V. create user

    # ejabberdctl register miguel softwarefools.com 'pass123'
    # ejabberdctl register niki softwarefools.com 'pass321'

### VI. open ports

    # iptables -I INPUT -p tcp --dport 5222 -j ACCEPT
    # iptables -I INPUT -p tcp --dport 5269 -j ACCEPT

TODO: in fact we need more here!

### VII. connect with client

* Get Pidgin http://www.pidgin.im/
* Connect & Enjoy 

### Further Work
* certificate
* dns configuration – should provide xmpp info (?


## Benchmark

### Tools for Stress Testing your RIG

    May 12, 2017

    Some nice tools to stress-test your computer:

    CPU: Prime95
    GPU: FurMark
    RAM: MemTest86+
    HDD/SSD: S.M.A.R.T

    Ref: http://www.pcworld.com/article/2028882/keep-it-stable-stupid-how-to-stress-test-your-pc-hardware.html

### Networking

    1. Connected my [laptop] and [desktop] via 5meter cat5e cable -> 1000mbit full duplex link
    2. created ~1.5G ramdisks with tmpfs on each.
    3. created a 1 giga file with data from /dev/urandom on [desktop]
    4. copied file with scp from [desktop] to [laptop]
    5. copied file with scp from [laptop] to [desktop]
    6. repeated 4 & 5 10 times : average speed ~70+MB/s from desk & 80+MB/s to desk, link stays up all the time. ping around 0.2-0.3ms
    7. repeated 1-6 with my other 3meter ethernet cable.
    8. tried both direction simultanously which gave a total data throughput even over 100MB/s

### Hard Disk

To get some information about the disk run:

Request identification info directly from the drive, which is displayed in a new expanded format with considerably more detail than with the older -i option.

    $ hdparm -I /dev/sda

Timings

    $ hdparm -tT /dev/sda

Clear Cache

    $ echo 3 > /proc/sys/vm/drop_caches 

I am using this block-size and count which results in ~2GB:

    $ dd [...] bs=1048576 count=2048

READ:

    $ dd if=test of=/dev/null
    $ dd if=/dev/sdX of=/dev/null

WRTIE:

    $ dd if=/dev/zero of=test
    $ dd if=/dev/zero of=/dev/sdX

smartmontools:

    seatage: 
    Raw_Read_Error_Rate
    Seek_Error_Rate  

    example:
    % python
    >>> 200009354607 & 0xFFFFFFFF
    2440858991 <---- total number
    >>> (200009354607 & 0xFFFF00000000) >> 32
    46 <--- number of errors

## Fixing HDD after Free Falling

So you also dropped your hard-disk and some of your secotrs got damaged,
probably by the head hitting the platter? The following short text outlines, 
how I approached fixing my disk after it hit the floor. Of course I would 
not rely on this disk for mission critical data anymore, but it 
might serve as a scratch/temporarily/redundant space.

Backup all your data in advance because the disk will almost certainly 
require reformatting and repartitoning after follwing this procedure!

Some Tools/Programms I used:

* smartctl
* hdparm
* grep

### smartctl

Try from fastest to slowest to find faulty sector: replace LBAi\_START,LBA\_END with meaningful values.

    smartctl -t [long|conveyence|short|select,LBA_START-LBA_END] -C /dev/sda 

The following did not work for me. Resulted in ‘interrupted by host’ for some reason. 
Probably the disks goes to sleep or something. Issueing some read operations every minute might fix this(?)

    -C to run test in foreground mode!

### hdparm

You can bi-search via –read-sector if complete ranges are affected (As in my case) and then fix them in one step:

    for i in {36312851..36312886}; do hdparm --write-sector $i /dev/sda; done

### Example

Exemplary session fixing one sector:

**WARNING: this will destroy your data !**

    1)  # smartctl -a /dev/sda | egrep 'Pend|Real|Offline_Unc'

      5 Reallocated_Sector_Ct   0x0033   100   100   036    Pre-fail  Always       -       4
    197 Current_Pending_Sector  0x0012   098   098   000    Old_age   Always       -       47
    198 Offline_Uncorrectable   0x0010   098   098   000    Old_age   Offline      -       47

    2)  # smartctl -t short /dev/sda

    3)  # smartctl -l selftest /dev/sda

    [...] 1  Short offline       Completed: read failure       90%     10632         152076520

    4)  # hdparm --read-sector 152076520 /dev/sda  # verification

    [...] reading sector 152076520: FAILED: Input/output error
    
    5)  # hdparm --write-sector 152076520 /dev/sda 

    6)  # smartctl -a /dev/sda | egrep 'Pend|Real|Offline_Unc'

      5 Reallocated_Sector_Ct   0x0033   100   100   036    Pre-fail  Always       -       4
    197 Current_Pending_Sector  0x0012   098   098   000    Old_age   Always       -       46
    198 Offline_Uncorrectable   0x0010   098   098   000    Old_age   Offline      -       46

## Raid and LVM


    Also refer to: http://www.tldp.org/LDP/Linux-Filesystem-Hierarchy/html/foreward.html
    
    Here I will write down some of my considarations about setting up a RAID
    and LVM.
    
    We want to get a performance boost and minimize the ssd wear-out.
    Also we will not put there any mission-critical files, because the content
    of the ssd will not be mirrored. beside regular backups of course.
    We also need to take the limited diskspace into account.
    while our hdd’s are 2*2TB we only have a single 250MB sdd.
    
    So we will be looking at files that are:
    
    * not updated very often
    * read often
    * not critical if they get lost.
    * not too huge
    
    So I consider for my usecase
    * /bin
    * /sbin
    * /boot
    * /lib{,32,64}
    
    * /usr (without /usr/local)
    
    That directories will reside on our raid1 since they contain important files:
    * /home/miguel
    * /root
    * /opt
    * /usr/local
    
    Finally we will mount the following things in ram for superfast and temporary space:
    
    * /tmp
    * /var/log (unless you care about logs)
    
    What about that directories?
    consider speed vs. security vs. drive wear-out
    
    * /etc
    * /var
    
    Never mount swap on your ssd!
    
    == extending logical volume and fs ==
    * https://wiki.archlinux.org/index.php/LVM
    
    # lvextend -L +20G VolGroup00/lvolhome
    
    resize fs (ext2, ext3 or ext4)
    for ext4 this can be done even without unmounting the device
    
    # resize2fs /dev//

## Some RAID Problems on my old Desktop

    https://www.thomas-krenn.com/en/wiki/Mdadm_checkarray
    https://plone.lucidsolutions.co.nz/linux/io/ssd-on-nvidia-sata-port-generates-error-eh-in-swncq-mode-and-failed-command-read-fpdma-queued/view
    https://www.howtoforge.com/replacing_hard_disks_in_a_raid1_array

    Kernel options: sata_nv.swncq=0

    [ 8625.896029] ata5.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
    [ 8625.896035] ata5.00: failed command: WRITE DMA EXT
    [ 8625.896040] ata5.00: cmd 35/00:08:50:06:b0/00:00:13:00:00/e0 tag 0 dma 4096 out
                            res 40/00:00:00:4f:c2/00:00:00:00:00/40 Emask 0x4 (timeout)
    [ 8625.896043] ata5.00: status: { DRDY }
    [ 8625.896048] ata5: hard resetting link
    [ 8625.896050] ata5: nv: skipping hardreset on occupied port
    [ 8626.364038] ata5: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
    [ 8631.372026] ata5.00: qc timeout (cmd 0x27)
    [ 8631.372031] ata5.00: failed to read native max address (err_mask=0x4)
    [ 8631.372033] ata5.00: HPA support seems broken, skipping HPA handling
    [ 8631.372035] ata5.00: revalidation failed (errno=-5)
    [ 8631.372042] ata5: hard resetting link
    [ 8631.372044] ata5: nv: skipping hardreset on occupied port
    [ 8631.840045] ata5: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
    [ 8631.864140] ata5.00: configured for UDMA/133
    [ 8631.864146] ata5.00: device reported invalid CHS sector 0
    [ 8631.864154] ata5: EH complete
    [ 8662.888029] ata5.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
    [ 8662.888036] ata5.00: failed command: WRITE DMA EXT
    [ 8662.888041] ata5.00: cmd 35/00:08:50:06:b0/00:00:13:00:00/e0 tag 0 dma 4096 out
                            res 40/00:00:00:4f:c2/00:00:00:00:00/40 Emask 0x4 (timeout)
    [ 8662.888043] ata5.00: status: { DRDY }
    [ 8662.888049] ata5: hard resetting link
    [ 8662.888051] ata5: nv: skipping hardreset on occupied port
    [ 8663.356038] ata5: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
    [ 8663.396135] ata5.00: configured for UDMA/133
    [ 8663.396141] ata5.00: device reported invalid CHS sector 0
    [ 8663.396151] ata5: EH complete
    [ 8693.864031] ata5.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
    [ 8693.864038] ata5.00: failed command: WRITE DMA EXT
    [ 8693.864043] ata5.00: cmd 35/00:08:50:06:b0/00:00:13:00:00/e0 tag 0 dma 4096 out
                            res 40/00:00:00:4f:c2/00:00:00:00:00/40 Emask 0x4 (timeout)
    [ 8693.864045] ata5.00: status: { DRDY }
    [ 8693.864051] ata5: hard resetting link
    [ 8693.864052] ata5: nv: skipping hardreset on occupied port
    [ 8694.332036] ata5: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
    [ 8694.356134] ata5.00: configured for UDMA/133
    [ 8694.356141] ata5.00: device reported invalid CHS sector 0
    [ 8694.356150] ata5: EH complete
    [ 8724.840030] ata5: limiting SATA link speed to 1.5 Gbps