Miscellanous Notes ================== A collection of short notes on things I wish to remember. Many topics are heavily outdated. Backing up and Restoring Docker Volumes --------------------------------------- July 3, 2017 One of my first questions, when starting my docker experience was how to backup and restore docker volumes. ### On the source host First of all identify your volumes: $ docker ps $ docker inspect my_container_name | grep -C 1 -i Source Now you can backup the directory to a tar on your host machine: $ docker run –rm –volumes-from my_container_name -v $(pwd):/backup ubuntu tar cvf /backup/backup_db.tar /var/lib/mysql ### On the target host In order to securely copy and restore the volume on the target host one can run: $ scp source_host:~/backup_db.tar . $ docker run –rm –volumes-frommy_container_name -v $(pwd):/backup ubuntu bash -c “cd /var/lib/mysql && tar xvf /backup/backup_db.tar –strip 1″ A Growing Collection of Linux Command Line One-Liners ------------------------------------------------------ Please believe me... this collection was really supposed to grow over time... inside a direcotry show disk usage of all hidden files and directories and sort by size: $ du $(ls .* -d | tail -n +3) -hs |sort -h inside a direcotry show disk usage of all files and directories (also hidden) and sort by size. Exclude ./DATA file. $ du . -a -d 1 -h –exclude=’./DATA’ | sort -h Tar all files in current directory, excluding ./DATA and ./.cache $ tar –exclude=’./.cache’ –exclude=’./DATA’ -cvf home_miguel_20180216.tar . Find files in ./ARCHIVE NOT belonging to a specific user: miguel $ find ARCHIVE/ \! -user miguel set folder/ permissions to Read/Browse only for owner recursively $ sudo chmod -R u=r,g=,o= folder/ $ chmod -R u=rX,g=,o= folder/ find all mails from Boban when in the maildir full of mailboxes and print only short headers without bodies: $ grepmail -H -B -Y ‘(^TO:|^From:)’ Boban * not really a one-lier but will print 256 colors in a bash: for i in {0..255} ; do printf "\x1b[48;5;%sm%3d\e[0m " "$i" "$i" if (( i == 15 )) || (( i > 15 )) && (( (i-15) % 6 == 0 )); then printf "\n"; fi done Some Tools of Choice -------------------- ### Systems * joomla * redaxo * typo3 * wordpress * mediawiki * oscommerce * opencart * owncloud * alfresco * mantis * piwik / matomo * loganalyzer * goaccess * nagios / icinga ### Servers * apache * nginx * mariadb / mysql * postfix * postgis * geoserver Compilation Notes ----------------- ### build your own webkit March 14, 2018 Let’s compile a release with debug info and install to /usr/local ~~~~~~ {.bash} wget https://webkitgtk.org/releases/webkitgtk-2.20.0.tar.xz tar -xvf webkitgtk-2.20.0.tar.xz cd webkitgtk-2.20.0 # install all the libs that will be reported missing in the next step. # I could not find the woff2 stuff in debian so skipped it... cmake -DPORT=GTK -DCMAKE_BUILD_TYPE=RelWithDebInfo -DUSE_WOFF2=NO -GNinja # this takes about 30minutes on my i7-4790K .. zzzzz..zzz ninja sudo ninja install ~~~~~~~~~~~ pkg-config pkg-config uses our new build now: [1] https://trac.webkit.org/wiki/BuildingGtk [2] https://webkitgtk.org/ ### Build and Install GCC April 27, 2015 #### Preparations Read the prequisites at [https://gcc.gnu.org/install/prerequisites.html](https://gcc.gnu.org/install/prerequisites.html) and get (most recent versions at time of this writing) the following: * gcc (5.2.0) * binutils (2.25.1) #### Configure and Build * unpack binutils-x.y.z * create a new directory binutils-x.y.z-build and inside it run the following commands: ~~~~~~ {.bash} $ ../binutils-x.y.z/configure --disable-nls --with-sysroot --enable-targets=all $ make -j4 $ make install ~~~~~~~~~~~~ * unpack gcc-x.y.z and run the contrib/download_prerequisites script inside. * create a new directory: gcc-x.y.z-build and inside it run: ~~~~~~ {.bash} $ ../gcc-x.y.z/configure --disable-nls --enable-languages=c,c++ --enable-threads $ make -j4 $ make install ~~~~~~~~~~~~ #### Reference [1] binutils and gcc README files. [2] https://gcc.gnu.org/install/ [3] http://wiki.osdev.org/Building_GCC [4] http://stackoverflow.com/questions/1726042/recipe-for-compiling-binutils-gcc-together monad transformers in action ---------------------------- January 1,2018 *Main Control.Monad.Writer Control.Monad.State> runState (runWriterT (get >>= \a -> tell ["foo"] >> put (a*a) >> tell ["bar"] >> tell [show a])) 5 emscripten ---------- October 1,2017 sdl2 port https://github.com/juj/emsdk android-ndk-r15c * accomplish build sys with: sdl2+input+audio+opengl+SDL_net+SDL_thread @ linux, win, osx, ios, android, win-phone, steam, rasp LUKS container over sshfs ------------------------- July 5, 2017 * enable user\_allow\_other in /etc/fuse.conf * sshfs -o allow\_root user@server:/BACKUPS/ ~/mnt/ * dd if=/dev/urandom of=~/mnt/megaloman bs=1M count=512 * sudo cryptsetup -y luksFormat ~/mnt/megaloman * sudo cryptsetup luksOpen ~/mnt/megaloman vol1 * sudo mkfs.ext4 /dev/mapper/vol1 * sudo mount /dev/mapper/vol1 /mnt * df -h | grep vol1 * sudo umount /mnt * sudo cryptsetup luksClose vol1 * fusermount -u ~/mnt NOTE: backup your data example use : rsync -a –info=progress2 source dist some areas of interest ---------------------- December 20, 2014 Some General Topics I am reasearching right now (or plan to do it). Or used to reasearch... * Neuronal Networks * Deep Learning * Random Forrests (and other techniques based on Decision Trees) * Simmulated Annealing * (Linear) Integer Programming Inter Process Communication --------------------------- March 14, 2018 We can attach nicely to same memory segment from 2 different processes: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ {.c .numberLines} // ipc via shared mem // attach to shared memory; key_t my_ftok = ftok("~/surf-webext-dom-shared-mem",'a'); int mem_seg=shmget(my_ftok,1024*1024,IPC_CREAT|0660); if(mem_seg==-1) { g_print("shmget failed: %s\n",strerror(errno)); } shared_buf=shmat(mem_seg,NULL,0); if(shared_buf==(void*)-1) { g_print("shmat failed: %s\n",strerror(errno)); } g_print("attached to shared memory.\n"); ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ SSL Certificate CSR ------------------- February 20, 2015 for apache ssl mod first check your old csr if you like: $ openssl req -text -noout -verify -in CSR.csr 1. generate key $ openssl genrsa -des3 -out www.yourdomain-example.com.key 2048 2. create CSR $ openssl req -new -key www.yourdomain-example.com.key -out www.yourdomain-example.com.csr refs: [1] https://search.thawte.com/support/ssl-digital-certificates/index?page=content&id=AR1108&actp=LIST Competitive Coding ------------------ March 17, 2015 A small selection of websites, which I visit from time to time, addressing competitive coding. * topcoder.com * codeforces.com * codility.com * codersclan.ne * projecteuler.net * www.codingame.com (need to check this) * www.codechef.com (need to check this) * http://psyho.gg/overview-of-programming-contests/ (and read here) * https://www.hackerrank.com/ * http://code.google.com/codejam * https://www.kaggle.com/ – predicitve modelling * odesk.com / elance Comp Science Literature ----------------------- March 17, 2015 This is a list of some books, focusing on topics around computer science, which I recently read or am currently reading or want to read :P * Meyers, Effective C++ (Addison-Wesley) C++98 only? * Meyers, More Effective C++ (Addison-Wesley) C++98 only? * Meyers, Effective STL. C++98 only? * Sutter, Exceptional C++ * Sutter, More Exceptional C++ * Karlson, Beyond the C++ Standard Library: An Introduction to Boost 1st Edition * Maybe Something on Multiprocessing (?) * Donald Knuth. The Art Of Computer Programming * Algorithms 3rd Edition (by Cormen, Leiserson, Rivest, Stein) * Jonathan Bartlett. Programming from the Ground Up * C++ in a Nutshell (O’REILLEY, by Lischner) * C++ Primer (5th Edition) by Lippman, Lajoie, Moo * The C++ Programming Language 4th Edition * Anthony Williams, C++ Concurrency in Action: Practical Multihreading * Dive into Python (2 and 3) by Mark Pilgrim * Learn You a Haskell for Great Good by Marian Lipovaca * Real World Haskell * Version Control with Git (O’REILLEY, by Loeliger & McCullough) (next: 9(10) ?) * The Linux Command Line, by William Shotts (http://linuxcommand.org/tlcl.php) * Debian, The Administrators Handbook (by Hertzog and Mas) * Absolute FreeBSD by Michael W. Lucas * Modern Operating Systems, Third Edition (by Andrew S. Tannebaum) * Linux Device Drivers (O’REILLEY, by Corbert, Rubini, Kroah-Hartman) * Linux Kernel in a Nutshell (O’REILLEY, Greg Kroah-Hartman) * Concrete Math (2nd) * UPENN cis194 * STOC '83 Proceedings of the fifteenth annual ACM symposium on Theory of computing / Primitives for the manipulation of general subdivisions and the computation of Voronoi diagrams * http://haskellbook.com/ * Wadler paper "Monads for Functional Programming" * simon marlow - book (parallel) * data 6 - course (Github) * sedgewick & waynes algorithms * books from appendix, from programming from ground up? VNC --- April 2, 2015 ### Server: $ apt-get install tightvncserver $ xtightvncserver -geometry 1900×1100 $ netstat -tap | grep vnc # remember port (e.g.5901) ### Client: $ ssh -L 5901:localhost:5901 you@remote # tunnel $ xtightvncviewer localhost::5901 # in new terminal ### Kill Server: $ vncserver -kill :1 # (or :2 etc..) Owncloud Notes -------------- ### Mounting Ownclouds DavFS April 15, 2015 $ apt-get install davfs2 * since owncloud seems to have problems with locks, edit /etc/davfs2/davfs2.conf and set use_locks to 0 in order to allow creating files (as well as probably writing) $ mount -t davfs https://secure.sf.com/owncloud/remote.php/webdav /mnt/ ### rescan/rebuild ownlocud files database June 22, 2016 inside ownlcoud directory run: sudo -u www-owncloud php console.php files:scan –all LXC --- April 28, 2015 maybe worth a try: apt-get install lxc; lxc create ctname -t download — -d debian -r jessie -a amd64; lxc-start -d -n ctname, lxc-attach ctname similar tools: chroot/ debootsrap Add vim Lang ------------ May 14, 2015 $ sudo apt-get install myspell-pl $ cd /ush/share/hunspell $ vim :mkspell pl pl_PL What every programmer should know about --------------------------------------- November 7, 2015 Undefined behaviour: http://blog.llvm.org/2011/05/what-every-c-programmer-should-know.html Memory: http://lwn.net/Articles/250967/ Floating point arithmetic: https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html IDE Comparison -------------- December 7, 2015 While I love working with vim (ctags, taglist, etc..) and the command line, it might be worth to have a look at the following IDE’s for Java and/or C++: * IntelliJ IDEA * Eclipse * Eclipse CDT * Codeblocks * Netbeans * Codelite * KDevelop Setup Postfix ------------- We want to handle different domains on a single postfix server and have virtual users along real unix users. For the virtual users we use uid/gid 5000. Relevant config files are: /etc/postfix/master.cf /etc/postfix/main.cf /etc/postfix/smtpd_sender_login_maps (tells which emails belong to which login) /etc/postfix/vmailbox (sepcifies our VIRTUAL mailboxes and the names of the spoolfiles) /etc/postfix/virtual (aliases for our local unix users) /etc/dovecot/users (specifies the logins, passwords, home directories,..) /etc/dovecot/conf.d/10-mail (set inbox and mailbox dirs) Be careful about the permissions of the different folders, so uid/gid 5000, postfix and dovecot can access as relevant. A very nice tutorial on setting up postfixadmin can be found here: https://lelutin.ca/posts/installing_postfix_-_clamav_-_spamassassin_-_dovecot_-_postfixadmin_on_debian_squeeze/ Windows Specific Notes ---------------------- ### disable hiberfile.sys September 7, 2017 run cmd.exe as administrator: powercfg.exe -h off ### GTA4 on NVIDIA-GTX 980 April 17, 2016 Add following flags : -nomemrestrict -norestriction Otherwise video memory is not identified correctly. ### Civilization 4 BTS Autosaves September 23, 2016 In the file: C:\Users\miguel\Documents\My Games\Beyond the Sword\CivilizationIV Adjust: ; The maximum number of autosaves kept in the directory before being deleted. MaxAutoSaves = 100 ; Specify the number of turns between autoSaves. 0 means no autosave. AutoSaveInterval = 1