Snippets

A blog post by the valuable.dev exploring creativity, fear, failure and learning.

Anything we try to create in this world will be a mix of fear, fun, and pain. The result will be either success, or regret.

Come back to this.

All that is gold does not glitter,
Not all those who wander are lost;
The old that is strong does not wither,
Deep roots are not reached by the frost.

From the ashes a fire shall be woken,
A light from the shadows shall spring;
Renewed shall be blade that was broken,
The crownless again shall be king.

J.R.R. Tolkien

Fan Ho (Chinese: 何藩; Jyutping: ho4faan4; 8 October 1931 – 19 June 2016) was a Chinese photographer, film director, and actor. From 1956, he won over 280 awards from international exhibitions and competitions worldwide for his photography

Robert Doisneau (French: [ʁɔbɛʁ dwano]; 14 April 1912 – 1 April 1994)[1] was a French photographer. From the 1930s, he photographed the streets of Paris. He was a champion of humanist photography and with Henri Cartier-Bresson a pioneer of photojournalism.

Doisneau is known for his 1950 image Le baiser de l'hôtel de khDoisneau is known for his 1950 image Le baiser de l'hôtel de ville (The Kiss by the City Hall), a photograph of a couple kissing on a busy Parisian street. He was appointed a Chevalier (Knight) of the Legion of Honour in 1984 by then French president, François Mitterrand.

A handy site with lots of tmux info, including how to open a pop-up window to fzf another session.

A site about career paths after reaching "Senior Software Engineer". Apparently you have two main branches, either engineering management, or staff engineer.

The site references several good books about the engineering management career path. I should come back to this.

archive

An article suggesting best practices for adding JSON output to CLI apps, and mentioning why JSON output is a good idea.

  • Do Make a Schema
  • Do Flatten the Structure
  • Do Output JSON Lines for Streaming Output
  • Do Use Predictable Key Names
  • Do Pretty Print with Two Spaces or Don’t Format at All
  • Don’t Use Special Characters in Key Names
  • Don’t Allow Duplicate Keys
  • Don’t Use Very Large Numbers

I've been trying to replace ALE with CoC. It didn't work. Maybe I have unusually high fixing expectations, but ALE is more capable and easier to setup. So I'll be using CoC and ALE together, which feels messy.

Formatting

  1. pip install pep8 autopep8
  2. <leader>fs
  • <leader>a - a list of all the wrong things

  • vif - select inside function

  • vam - select method

  • <leader>ac - see a box with options to fix imports or ignore file

fd is a replacement for find with more intuitive defaults.

  • Case-insensitive.
  • Ignores hidden directories and files.
  • Ignores patterns in .gitignore.

Commands

  • fd PATTERN
  • not find -iname '*PATTERN*'
  • fd --help

repo

  • List of sessions: <prefix> s

  • Last session: <prefix> Z

  • Rename session: <prefix> $

  • Rename window: <prefix> ,

  • Previous session: <prefix> (

For last session I used the following key bind:

bind Z switch-client -l
  • 3 copies, 2 types of media, 1 copy off-site
  • Have 1 primary backup and 2 duplicates, in additional to the original data.
  • NAS, USB drive, Cloud storage, Server, Glacier, etc

For example, your laptop could be backed up to a NAS, a server, and glacier storage. Or 2 NAS and 1 archival cloud storage.

Reverse proxy server

  • A reverse proxy is a server.
  • It sits in front of other servers and forwards client requests to them.
  • Is a middleman or broker on behalf of origin servers.

Usually used to:

  • Increase security (keeping the IP addresses of origin servers private makes DDoS attacks much harder).
  • Optimize performance (caching).
  • Increase reliability (load balancing).

The client thinks that the reverse proxy is the end-point server.

Normal proxies

  • Also called a web or forwarding (normal) proxy.
  • Sits between client machines and hosts.
  • Is a middleman on behalf of the clients.

The server thinks that the client is the proxy.

Source: cloudflare

  • If you can ssh into the server then Unison should work.
  • Install the same version on the client and on the server.

To sync the contents of dir b into dir a:

unison -batch -color true <dir-a> ssh://<user>@a<host>/<dir-b>

Setup

  1. Download the binary file with wget
  2. chmod +x &lt;file&gt;
  3. copy the executable somewhere on your path like /usr/local/bin
  4. unison -doc tutorial > remote method > remote shell method
  5. use a filewatcher to sync on change, or use cron to sync every n minutes

You can also use unison to sync files on the same computer.

References

gist

Background

Recently I needed to work on a codebase that was on a remote machine whilst still using my local workflow and developer tools.

Vim's built in scp:// functionality isn't versatile enough - i wanted to use fzf.vim to search for text across the code base. This led me to rsync, sshfs, and eventually unison.

Chisoku

Being content with what you already have. You don't need more stuff, or new stuff, just because it exists. You're being marketed at, or marked.

Do you need it or do you want it? Need is finite, want is infinite.

Wabi Sabi

Beauty in imperfection. As things begin to wear, they develop some character that can make them nicer than something that is fresh. It's the opposite of new shiny thing syndrome.

Mitate

An object has more than one purpose. Repurposing, inventing, adapting, etc is interesting and a mini achievement that can be celebrated.

Source.

An introductory article with examples from Smashing Magazine that introduces and explains what alpine.js is and how it is different to jQuery and vue.js.

source and exec are both built-ins.

source evaluates or runs the content of a file. For example, if you source ~/.zshrc you apply the content of the file to the currently running Zsh process. You can source and valid Zsh code.

exec replaces the current shell process with another process. Your terminal (tty) session is running a shell. Replace it with another shell without launching another tty. It could be the same shell with different settings or flags.

The tty (teletypewriter) command prints the name of the terminal you're using.

Chezmoi is a dotfiles manager.

  • chezmoi diff - see the difference between the current state of your local dotfiles compared against the checked-in versions.
  • cm aa - to update chezmoi with all your local changes.
alias -g aa="status | cut -c 4- | xargs -I % -p sh -c 'chezmoi add ~/%'"

In zsh you can suspend a command whilst typing it.

This clears the current line, lets you run another command, and then when it finishes the original unfinished text is put back.

This is useful if whilst typing a command you realise you need to run another command first, like:

  • ls to check the name for an argument.
  • creating a location before moving something into that location.
  • checking man or tldr for a command flag.

Zsh has built-in functions push-line and push-input.

blog post and also this SO answer

zsh widgets zsh documentation

Interesting article about writing down a list of manual steps for tasks that can't be automated. A do-nothing script can take some of the toil out of a manual slog.

Even though the script doesn't do anything other than display text on a screen (the instructions to complete the task) it's still helpful, because:

  • All the steps and information required are collected together.
  • It lowers the activation energy required to do something fiddly.
  • Each step is part of a function, and the function can be updated with automation code if automation becomes possible.
  • It's more difficult to lose your place in the process, or get the order of steps wrong.

It doesn't reduce manual effort. It does reduce toil.

Jack Daniels' book about running has a training plan that seems to be highly regarded.

training
  • LT1 - 75% of maximum heart rate. ~145BPM maybe (for running).
  • LT2 - Average heart rate for a maximum sustained effort for 1 hour.

Zone 1

  • Trains your body to use fat stores as energy.
  • Increases your oxidative capacity.
  • Trains slow-twitch muscle fibres.
  • You can speak a full sentence without taking a breath.

Zone 3

  • Trains your body to use carbohydrates as energy.
  • Trains your glycolytic system.
  • Increases your ability to mitigate lactic acid.
  • Trains fast-twitch muscle fibres.
  • You can barely speak in this zone.

Training

  • Zone 2 is not a good place to train, it's middle of the road and not an efficient use of time or effort.
  • 80% of training should be in zone 1.
  • We have much more energy stored as fat than we do as carbs.
  • 7 days/week
  • 25 km/day
  • Start easy. Last 50% begin to move through the field.
  • 40% - 50% of entrants DNF
  • 90% of the course is runnable
  • 50km: 6:10 min/km (Diez Vista 50km)
  • 50km: 7:25 - 8:00 min/km (Beaver Flat 50 JP)
  • 160km: 10:35 min/km (JP)
  • 150km: 6:30 - 8:00 min/km (UTMB CD)
  • training: 6:10 is slow enough to stay fresh. 5:10-5:30 is fast.

A project to demonstrate an offline, fast, point-of-sale device to allow merchants and customers to make fast cheap transactions on the lightning network.

Also lnbits

This article is a fairly practical introduction to the GNU make tool.

It's not the article I was looking for though, so I might swap it out if I find a better one.

Google Pub/Sub has client libraries in all the usual languages. You can also construct the API calls yourself. This is a link to the API documentation.

If I were to use a JavaScript beacon to push a message to a topic, my web analytics engine (a collection of cloud functions) could subscribe to the topic.

This would have the advantage that if the site were flooded with traffic and the maximum number of function instances wasn't enough to handle all the events, then none of the page views (or other events) would be lost because Pub/Sub guarantees delivery and would keep trying to deliver the message for up to 7 days.

Using the beacon to trigger the cloud functions directly wouldn't work at scale, because once the maximum number of instances are being triggered as frequently as the function takes to run, the endpoint would become unresponsive. There is no caching layer.

However, the site isn't being flooded with traffic, and I have better things to do than fix stuff that isn't broken.

A twitter thread showing how a business started creating handwritten notes to customers using a plotter, and how they scaled up and further automated the process.

It would be cool to create something like this for western Europe. Handwrytten.com seems to be the main commercial implementation of this idea in NA. Competitors.

I still think it would be fun to make a simple photo printing service. Share a photo in a WhatsApp message and its printed and delivered.

A ledger (what's a ledger? It's a webpage) from the "Center For Humane Technology" showing some of the dangers of social media.

The "Do unto others" section is outstanding. It describes the views of some tech company leaders, including how they limit usage of devices and social media for themselves and their kids.

If youth is wasted on the young, then life is wasted on the living.

This article about how people become slaves is very uncomfortable reading.

It puts a very different perspective on my life, ambitions and perceived risks.

Google Cloud Platform have introduced a "minimum instances" features in the Cloud Functions product.

If you have a function using 128MB of memory (the smallest option), it will cost a minimum €0.061 per day (€1.86 per month) to run 1 minimum instance.

sudo nmap -sP 192.168.1.1/24 | grep "Nmap"

Thanks to Jeff Geerling (again) I found this nifty command to see what devices are connected on a local network.

Found in this article about setting up a pi-hole.

Your relationships will come to define your life more than anything else.

A blog post about someone who read their dad's journal after he died. Its about regret, grief, and struggles against internal contradictions. Its about the tangible consequences of intangible relationships.

The comments are also significant.

Miller is like awk, sed, cut, join, and sort for data formats such as CSV, TSV, tabular JSON and positionally-indexed.

Try this if you're operating on JSON of CSV data, and you want to use column names instead of column numbers.

Organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations - Melvin Conway

In reverse:

If the architecture of the system and the architecture of the organization are at odds, the architecture of the organization wins - Ruth Malan

A blog post about how someone compromised a group of web apps.

It lists a series of technologies and techniques that the author uses as they progress their investigation.

These would make a useful list of things to know in order to build safe web-apps and not repeat the mistakes of the unfortunate target company.

  • I live and work near Amsterdam
  • My manager is in Berlin
  • A stakeholder is in Boston
  • Another stakeholder is in India
  • Another stakeholder is in Ireland

¯\_(ツ)_/¯

"Without psychological safety, respect, and trust, none of the following is possible"

"The best ways to work are collaborative. Negotiation is not collaboration. Isolated individuals making heroic efforts are never as effective as collaborative groups. We get the best results when customers, business people, and developers literally work together."

Blog post

Zwischenzugs blog post arguing that to achieve a significant change in an organisation you need need to:

  1. Get funding.
  2. Persuade the finance department to give you money.
  3. Understand what they value.
  4. Understand their cash flows.
  5. Understand how and why customers or clients part with their money.
  6. Understand business constraints (legal, reg, operating).

The five whys approach to problem solving

"Consider a deeper structural cause of cultural problems in change management: how money flows through the organisation."

"If you want to transform IT in an enterprise, start with finance. If you can crack that, you've a chance to succeed with sec and controls functions. If you don’t know why it’s important to start with finance, you’ll definitely fail "

Yesterday my daughter asked me to write a page in her Freundebuch.

Of all tyrannies, a tyranny sincerely exercised for the good of its victims may be the most oppressive. It would be better to live under robber barons than under omnipotent moral busybodies.

The robber baron's cruelty may sometimes sleep, his cupidity may at some point be satiated; but those who torment us for our own good will torment us without end for they do so with the approval of their own conscience. They may be more likely to go to Heaven yet at the same time likelier to make a Hell of earth.

This very kindness stings with intolerable insult. To be "cured" against one's will and cured of states which we may not regard as disease is to be put on a level of those who have not yet reached the age of reason or those who never will; to be classed with infants, imbeciles, and domestic animals.

- C. S. Lewis

Blog post about how to make it easier to upgrade a cryptographic or hashing library.

Django encodes passwords for database storage like this:

<algorithm>$<iterations>$<salt>$<hash>

Interestingly, Giovanni Collazo emphasises that we should design systems for change, which initially seems pretty close to contradicting YAGNI, but the answer lies in the context.

Contains too much conjecture at the start, but becomes more substantial.

At high levels of competition the difference between "good" and "great" is partly determined by how much pain you are willing to experience.

There are great benefits from training in a secondary discipline.

  • Anderson Silva is a ballet dancer.
  • Conor McGregor does "movement training".
  • Israel Adesanya dances competitively.
  • Georges St-Pierre practices gymnastics.

archive

Rabbits, fluid convection, the Mandelbrot set and lots of others things too. Also known as bifurcation theory.

The video shows how the Feigenbaum Constant is defined. It's a fundamental constant I hadn't come across before - 4.6692..

archive

article

"An Apple onion router. The routing uses two hops; Apple provides the first, and independent third parties (not yet specified) provide the second."

"In one move, Apple has taken onion routing from a specialized tool for hackers to something that will be in daily use."

An amateur practices until they can play it correctly, a professional practices until they can't play it incorrectly.

  • Invoked with s followed by 2 chars.
  • S goes backwards.
  • F, f, T, and t are enabled to work across lines.
  • Jump back with <C-O>
  • ; or , to go to next/previous highlighted text.
  • 5sxy searches for the next instance of xy within 5 lines.
  • 3dzqt delete up to the third instance of qt.

repo

A basic statement that is assumed to be true. E.g: "A straight line can be drawn between any two points"

archive

A lobster diver was swallowed by a humpback whale. Then it spit him out. Humpback whales don't have teeth, and have reduced forward vision when they open their mouths to feed.

article
archive

Key commands

  • git lfs install (also uninstall)
  • git lfs track "**/*.mp4"
  • git lfs ls-files
  • git lfs status

track just updates the .gitattributes file.

Commit the .gitattributes file with the tracking configuration before committing the large files.

status or ls-files should show the large files in question before you push the commit that starts tracking the large files.

Blog post demonstrating how to split a .bashrc file into "submodules" and keep it maintainable.

BASHRC_D=~/.config/bashrc.d
[[ -r ${BASHRC_D}/bootstrap ]] && . ${BASHRC_D}/bootstrap
for file in ${BASHRC_D}/*.sh; do
  [[ -r $file ]] && . $file
done
unset file

Something should happen within 100ms of the users input in order to maintain a feeling of responsiveness.

If something happens within 50ms of the trigger event, it will feel almost instant.

Also, checkout hyperfine for performance benchmarking.

Lots of useful tips in the original blog post.

  • [s or ]s → go to next/previous bad word
  • z= → list of suggestions
  • zg → add word to good word list
  • zug → remove word from good word list
  • zw → add word to bad word list
  • zuw → remove word from bad word list
grep -inr --include package.json \
    'shortcut": {' . -A 3
  • It's the --include flag that does the important part.
  • -i → case insensitive
  • -n → print line number
  • -r → recursive from starting page
  • . → start in current directory
  • -A 3 → print the 3 lines below the found line

:map → show a list of the current keyboard mappings for normal, visual, select and operator pending modes.

:map! → show a list of the current keyboard mappings for insert and command-line mode

Top put all the mappings into a convenient text file:

:redir! > vim_maps.txt
:map
:map!
:redir END

source another source

I know about ImprovMX, which used to be great because you could do a lot for free, but now you only get 1 domain for free.

ForwardEmail are 3 times cheaper than ImprovMX, and I have 2 domains forwarding email. It's not particularly private, but I can send and recieve from a domain, for free.

  • ???? → 4 chars

  • * → any number of chars

  • [:upper:][A-Z] same for [:lower:] and [:digit:]

  • [:alpha:][a-zA-Z]

  • [:alnum:][a-zA-Z0-9]

  • ls -l [a-d] → part of a range

  • ^ and $ works like in regex

  • la a*.{doc,docx} → OR

  • ls a*.(doc|docx) → OR

Create a custom command and function to create a new file in vim.

command! -nargs=1 Ms call s:NewFile(<q-args>)`

function! s:NewFile(fp)
  echom a:fp
  execute "e " . "~/foo/bar/" . a:fp . ".ext"
endfunction

Useful help:

  • :h %:h → filename modifiers
  • :h expand() → expand wildcards, including filename modifiers

question on SO

  • target function arguments
  • more types of object
  • consistent searching/jumping if you're not inside the thing you're targeting
  • jump forward or backward
  • look for the nth occurrence
  • select white space around/inside object

Github
Article about a plugin

  • <C-h> - same as backspace
  • <C-w> - delete previous word
  • <C-u> - delete everything before cursor (on same row)
  • <C-d> or <C-t> - (un)indent a row
  • <C-e> - delete next word (create a mapping in vimrc)

It's annoying when you delete something and overwrite your yanked text.

Use numbered registers! "0 to "9

  • "0 contains the most recent yank.
  • "1 contains the most recent deleted text
  • "0p - paste the most recent yank, even if you deleted something after yanking it

Chezmoi is a great tool for managing dotfiles. This is a shortcut to update the source state based on local changes.

chezmoi status | cut -c 4- | xargs -I % -p sh -c 'chezmoi add ~/%'

Github

“It enhances your memory and makes you more creative. It makes you look more attractive. It keeps you slim and lowers food cravings. It protects you from cancer and dementia. It wards off colds and the flu. It lowers your risk of heart attacks and stroke, not to mention diabetes. You’ll even feel happier, less depressed, and less anxious.”

Why We Sleep by Dr. Matt Walker

From an Interview:

I don't want to claim that programming is an art, because it really is mostly just about 'good engineering'. I'm a big believer in Thomas Edison's 'one percent inspiration and ninety-nine percent perspiration' mantra. It's almost all about the little details and the everyday grunt-work.

But there is that occasional 'inspiration' part, that 'good taste' thing that is about more than just solving some problem - solving it cleanly and nicely and yes, even beautifully.

A RPC is when an executable causes a procedure (subroutine) to execute on another computer, It's coded as if it were a normal (local) subroutine. You don't explicitly code the details for the remote interaction. You write the same code whether the subroutine is local or remote.

lsblk is a command to get info about connected devices.

Used when attaching drives.

Make aliased files the real file

for f in $(find . -type l -maxdepth 1);
    do cp --remove-destination $(readlink $f) $f;
done;

Save (write) a (read only) Vim file with sudo when you opened it without sudo-ing:

:w !sudo tee % > /dev/null

Broot is a tool that shows the contents of a directory on one screen, even its got lots of files or sub-directories.

I should see if I can use GitHub actions to generate html from markdown and run some shell and python scripts.

I was running a binary in Debian that was complaining about an environment variable not existing. I moved the binary into a $PATH directory and logged in as a sudo user. Why did this solve the problem?

As a hacker, or creator, or whatever the best label is, I always want to create something (usually code) and have it finished.

But a strange creativity and productivity boost comes from dabbling, dipping in and out.

I think that if the technical challenges aren't too hard, then the main criteria for success is creativity.

Creativity needs time away from the project, and sleep, to bubble up and let ideas grow.

Ultimately, the most successful path is usually the most interesting, because success has more consequences than failure. "Interesting" requires elements of novelty and surprise, and without creativity these elements can't flourish. 3/n

Dabbling results in more creativity than 6+ hours of strenuous work, and is more likely to give you satisfying results.

bar << foo bar will stop reading input when it reached "foo".

bar <<< "foo" foo is all the input. bar wont run interactively.

bar < <(foo:list) process subscription. Kind of like piping in the output of multiple commands.

Stack Overflow

I tried to read and write the same file in a pipeline, and got caught out by a race condition (why is the file empty?!). Do this instead:

some_script < file > smscrpt.$$ \
&& mv smscrpt.$$ file || rm smscrpt.$$

|| removes the temporary file if it errors.

$$ is the process ID and ensures that you always have a unique temporary file name.

jj - A stream editor
jq - A json processor
python -m json.tool

I like jq for pretty printing JSON output, jj for making JSON pretty or condensed.

This was really useful when optimizing the search index for this blog.

docker run -d ...
docker logs -f <ID>
docker run -it ...
docker run -itd
docker container attach <ID>
<C-p><C-q> -> detach from container interactively

stack overflow

Split long strings (or command outputs) onto multiple lines Find and replace a particular char (maybe :) with a \n.

... | tr ':' '\n'

... | sed 's/:/\n/g'
  1. Without noticing, create a .gitignore file with a single * in it.
  2. Spend a day trying to understand why ripgrep has stopped working for only 1 project.
  3. 😭 😭 😭

If you're cat-ing a file and the bash prompt doesn't start on a new line (cos the file you displayed using cat doesn't end with a new line char) the following will fix it:

cat <filename> ; echo

Cloning large repos, or repos with large files in them, doesn't work with git clone ... you need to use git lfs clone ...

So why is git lfs clone deprecated? What's the replacement?

Use nohup to keep a curl process running even when the terminal (tty?) session autocloses at 3am.

You can group shells into groups:

  1. ksh - korn shell and zshell
  2. sh - bourne shell and bash (the bourne again shell)

Because

  • zsh isn't a superset of bash.
  • bash is a superset of the bourne shell.