BlogCollectionsPhotographsSnippetsMetricsAboutChat
Categories:
Snippets
Notes and bookmarks to content from around the web.

Another excellent and interesting blog post from Simon Willison about OpenAI's DALL-E 3 text to image AI.

  • generating consistent characters
  • modifying images
  • getting the seed for an image
  • details about the prompt that chatGPT uses when sending the users prompt to the DALL-E model.

A paper describing what the unrestricted GPT-4 can do.

Come back to this

Can do:

  • theory of mind
  • use tools and external APIs
  • abstract and creative tasks like mathlete or fermi questions

Can't do:

  • Discontinuous tasks - tasks that require knowing the end of the answer when the answer begins.
Investors are keeping their coins still, with the amount of Bitcoin not budging nearing a new all-time high.

A Decrypt article reporting on research by Glassnode.

  1. Glassnode added in the report that there is “a turning of the cycle and a marked shift in investor behavior patterns.”
  2. such activity has been previously observed in prior bear markets, “potentially signaling a perception that the market is oversold”
  3. Dormant coins actually become “increasingly unlikely to be spent” after a 155-day holding period

A blog post by Stephen Bailey articulating ideas I've thought about but never seen anyone else consider;

  1. An aggressive1 attitude can drive progress - it's one of the mental tools in the toolbox.
  2. Software engineering is a craft - I am a craftsman. Not an artist, and not a technician.

The active and violent spirit of the postdoc is a critical difference in the ways that early- and mid-career professionals grow in their craft.

It takes animus to push conversations out from the boring straits of explanation and pedantry and into the frothy, conflicting waves of exploration and world expansion. It takes a bit of pirate swagger. It takes a separation of the world into mateys and landlubbers.

After the beginner’s scaffolding of skills, what matters is the mechanism for deciding what’s next. That’s where opinion and aversion — not reason! — comes in.

Footnotes

  1. And therefore opinionated

A handy site with lots of tmux info, including how to open a pop-up window to fzf another session.

A site about career paths after reaching "Senior Software Engineer". Apparently you have two main branches, either engineering management, or staff engineer.

The site references several good books about the engineering management career path. I should come back to this.

archive

An article suggesting best practices for adding JSON output to CLI apps, and mentioning why JSON output is a good idea.

  • Do Make a Schema
  • Do Flatten the Structure
  • Do Output JSON Lines for Streaming Output
  • Do Use Predictable Key Names
  • Do Pretty Print with Two Spaces or Don’t Format at All
  • Don’t Use Special Characters in Key Names
  • Don’t Allow Duplicate Keys
  • Don’t Use Very Large Numbers

I've been trying to replace ALE with CoC. It didn't work. Maybe I have unusually high fixing expectations, but ALE is more capable and easier to setup. So I'll be using CoC and ALE together, which feels messy.

Formatting

  1. pip install pep8 autopep8
  2. <leader>fs
  • <leader>a - a list of all the wrong things

  • vif - select inside function

  • vam - select method

  • <leader>ac - see a box with options to fix imports or ignore file

fd is a replacement for find with more intuitive defaults.

  • Case-insensitive.
  • Ignores hidden directories and files.
  • Ignores patterns in .gitignore.

Commands

  • fd PATTERN
  • not find -iname '*PATTERN*'
  • fd --help

repo

  • Sort sessions alphabetically <prefix> s then O

  • List of sessions: <prefix> s

  • Last session: <prefix> Z

  • Rename session: <prefix> $

  • Rename window: <prefix> ,

  • Previous session: <prefix> (

For last session I used the following key bind:

bind Z switch-client -l
  • 3 copies, 2 types of media, 1 copy off-site
  • Have 1 primary backup and 2 duplicates, in additional to the original data.
  • NAS, USB drive, Cloud storage, Server, Glacier, etc

For example, your laptop could be backed up to a NAS, a server, and glacier storage. Or 2 NAS and 1 archival cloud storage.

Reverse proxy server

  • A reverse proxy is a server.
  • It sits in front of other servers and forwards client requests to them.
  • Is a middleman or broker on behalf of origin servers.

Usually used to:

  • Increase security (keeping the IP addresses of origin servers private makes DDoS attacks much harder).
  • Optimize performance (caching).
  • Increase reliability (load balancing).

The client thinks that the reverse proxy is the end-point server.

Normal proxies

  • Also called a web or forwarding (normal) proxy.
  • Sits between client machines and hosts.
  • Is a middleman on behalf of the clients.

The server thinks that the client is the proxy.

Source: cloudflare

  • If you can ssh into the server then Unison should work.
  • Install the same version on the client and on the server.

To sync the contents of dir b into dir a:

unison -batch -color true <dir-a> ssh://<user>@a<host>/<dir-b>

Setup

  1. Download the binary file with wget
  2. chmod +x &lt;file&gt;
  3. copy the executable somewhere on your path like /usr/local/bin
  4. unison -doc tutorial > remote method > remote shell method
  5. use a filewatcher to sync on change, or use cron to sync every n minutes

You can also use unison to sync files on the same computer.

References

gist

Background

Recently I needed to work on a codebase that was on a remote machine whilst still using my local workflow and developer tools.

Vim's built in scp:// functionality isn't versatile enough - i wanted to use fzf.vim to search for text across the code base. This led me to rsync, sshfs, and eventually unison.

An introductory article with examples from Smashing Magazine that introduces and explains what alpine.js is and how it is different to jQuery and vue.js.

source and exec are both built-ins.

source evaluates or runs the content of a file. For example, if you source ~/.zshrc you apply the content of the file to the currently running Zsh process. You can source any valid Zsh code.

exec replaces the current shell process with another process. Your terminal (tty) session is running a shell. Replace it with another shell without launching another tty. It could be the same shell with different settings or flags.

The tty (teletypewriter) command prints the name of the terminal you're using.

Chezmoi is a dotfiles manager.

  • chezmoi diff - see the difference between the current state of your local dotfiles compared against the checked-in versions.
  • cm aa - to update chezmoi with all your local changes.
alias -g aa="status | cut -c 4- | xargs -I % -p sh -c 'chezmoi add ~/%'"

In zsh you can suspend a command whilst typing it.

This clears the current line, lets you run another command, and then when it finishes the original unfinished text is put back.

This is useful if whilst typing a command you realise you need to run another command first, like:

  • ls to check the name for an argument.
  • creating a location before moving something into that location.
  • checking man or tldr for a command flag.

Zsh has built-in functions push-line and push-input.

Interesting article about writing down a list of manual steps for tasks that can't be automated. A do-nothing script can take some of the toil out of a manual slog.

Even though the script doesn't do anything other than display text on a screen (the instructions to complete the task) it's still helpful, because:

  • All the steps and information required are collected together.
  • It lowers the activation energy required to do something fiddly.
  • Each step is part of a function, and the function can be updated with automation code if automation becomes possible.
  • It's more difficult to lose your place in the process, or get the order of steps wrong.

It doesn't reduce manual effort. It does reduce toil.

A project to demonstrate an offline, fast, point-of-sale device to allow merchants and customers to make fast cheap transactions on the lightning network.

Also lnbits

This article is a fairly practical introduction to the GNU make tool.

It's not the article I was looking for though, so I might swap it out if I find a better one.

Google Pub/Sub has client libraries in all the usual languages. You can also construct the API calls yourself. This is a link to the API documentation.

If I were to use a JavaScript beacon to push a message to a topic, my web analytics engine (a collection of cloud functions) could subscribe to the topic.

This would have the advantage that if the site were flooded with traffic and the maximum number of function instances wasn't enough to handle all the events, then none of the page views (or other events) would be lost because Pub/Sub guarantees delivery and would keep trying to deliver the message for up to 7 days.

Using the beacon to trigger the cloud functions directly wouldn't work at scale, because once the maximum number of instances are being triggered as frequently as the function takes to run, the endpoint would become unresponsive. There is no caching layer.

However, the site isn't being flooded with traffic, and I have better things to do than fix stuff that isn't broken.

A twitter thread showing how a business started creating handwritten notes to customers using a plotter, and how they scaled up and further automated the process.

It would be cool to create something like this for western Europe. Handwrytten.com seems to be the main commercial implementation of this idea in NA. Competitors.

I still think it would be fun to make a simple photo printing service. Share a photo in a WhatsApp message and its printed and delivered.

A ledger (what's a ledger? It's a webpage) from the "Center For Humane Technology" showing some of the dangers of social media.

The "Do unto others" section is outstanding. It describes the views of some tech company leaders, including how they limit usage of devices and social media for themselves and their kids.

Google Cloud Platform have introduced a "minimum instances" features in the Cloud Functions product.

If you have a function using 128MB of memory (the smallest option), it will cost a minimum €0.061 per day (€1.86 per month) to run 1 minimum instance.

sudo nmap -sP 192.168.1.1/24 | grep "Nmap"

Thanks to Jeff Geerling (again) I found this nifty command to see what devices are connected on a local network.

Found in this article about setting up a pi-hole.

Miller is like awk, sed, cut, join, and sort for data formats such as CSV, TSV, tabular JSON and positionally-indexed.

Try this if you're operating on JSON of CSV data, and you want to use column names instead of column numbers.

Organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations - Melvin Conway

In reverse:

If the architecture of the system and the architecture of the organization are at odds, the architecture of the organization wins - Ruth Malan

A blog post about how someone compromised a group of web apps.

It lists a series of technologies and techniques that the author uses as they progress their investigation.

These would make a useful list of things to know in order to build safe web-apps and not repeat the mistakes of the unfortunate target company.

"Without psychological safety, respect, and trust, none of the following is possible"

"The best ways to work are collaborative. Negotiation is not collaboration. Isolated individuals making heroic efforts are never as effective as collaborative groups. We get the best results when customers, business people, and developers literally work together."

Blog post

Zwischenzugs blog post arguing that to achieve a significant change in an organisation you need need to:

  1. Get funding.
  2. Persuade the finance department to give you money.
  3. Understand what they value.
  4. Understand their cash flows.
  5. Understand how and why customers or clients part with their money.
  6. Understand business constraints (legal, reg, operating).

The five whys approach to problem solving

"Consider a deeper structural cause of cultural problems in change management: how money flows through the organisation."

"If you want to transform IT in an enterprise, start with finance. If you can crack that, you've a chance to succeed with sec and controls functions. If you don’t know why it’s important to start with finance, you’ll definitely fail "

Blog post about how to make it easier to upgrade a cryptographic or hashing library.

Django encodes passwords for database storage like this:

<algorithm>$<iterations>$<salt>$<hash>

Interestingly, Giovanni Collazo emphasises that we should design systems for change, which initially seems pretty close to contradicting YAGNI, but the answer lies in the context.

article

"An Apple onion router. The routing uses two hops; Apple provides the first, and independent third parties (not yet specified) provide the second."

"In one move, Apple has taken onion routing from a specialized tool for hackers to something that will be in daily use."

  • Invoked with s followed by 2 chars.
  • S goes backwards.
  • F, f, T, and t are enabled to work across lines.
  • Jump back with <C-O>
  • ; or , to go to next/previous highlighted text.
  • 5sxy searches for the next instance of xy within 5 lines.
  • 3dzqt delete up to the third instance of qt.

repo

Key commands

  • git lfs install (also uninstall)
  • git lfs track "**/*.mp4"
  • git lfs ls-files
  • git lfs status

track just updates the .gitattributes file.

Commit the .gitattributes file with the tracking configuration before committing the large files.

status or ls-files should show the large files in question before you push the commit that starts tracking the large files.

Blog post demonstrating how to split a .bashrc file into "submodules" and keep it maintainable.

BASHRC_D=~/.config/bashrc.d
[[ -r ${BASHRC_D}/bootstrap ]] && . ${BASHRC_D}/bootstrap
for file in ${BASHRC_D}/*.sh; do
  [[ -r $file ]] && . $file
done
unset file

Something should happen within 100ms of the users input in order to maintain a feeling of responsiveness.

If something happens within 50ms of the trigger event, it will feel almost instant.

Also, checkout hyperfine for performance benchmarking.

Lots of useful tips in the original blog post.

  • [s or ]s → go to next/previous bad word
  • z= → list of suggestions
  • zg → add word to good word list
  • zug → remove word from good word list
  • zw → add word to bad word list
  • zuw → remove word from bad word list
grep -inr --include package.json \
    'shortcut": {' . -A 3
  • It's the --include flag that does the important part.
  • -i → case insensitive
  • -n → print line number
  • -r → recursive from starting page
  • . → start in current directory
  • -A 3 → print the 3 lines below the found line

:map → show a list of the current keyboard mappings for normal, visual, select and operator pending modes.

:map! → show a list of the current keyboard mappings for insert and command-line mode

Top put all the mappings into a convenient text file:

:redir! > vim_maps.txt
:map
:map!
:redir END

source another source

I know about ImprovMX, which used to be great because you could do a lot for free, but now you only get 1 domain for free.

ForwardEmail are 3 times cheaper than ImprovMX, and I have 2 domains forwarding email. It's not particularly private, but I can send and recieve from a domain, for free.

  • ???? → 4 chars

  • * → any number of chars

  • [:upper:][A-Z] same for [:lower:] and [:digit:]

  • [:alpha:][a-zA-Z]

  • [:alnum:][a-zA-Z0-9]

  • ls -l [a-d] → part of a range

  • ^ and $ works like in regex

  • la a*.{doc,docx} → OR

  • ls a*.(doc|docx) → OR

Create a custom command and function to create a new file in vim.

command! -nargs=1 Ms call s:NewFile(<q-args>)`

function! s:NewFile(fp)
  echom a:fp
  execute "e " . "~/foo/bar/" . a:fp . ".ext"
endfunction

Useful help:

  • :h %:h → filename modifiers
  • :h expand() → expand wildcards, including filename modifiers

question on SO

  • target function arguments
  • more types of object
  • consistent searching/jumping if you're not inside the thing you're targeting
  • jump forward or backward
  • look for the nth occurrence
  • select white space around/inside object

Github
Article about a plugin

  • <C-h> - same as backspace
  • <C-w> - delete previous word
  • <C-u> - delete everything before cursor (on same row)
  • <C-d> or <C-t> - (un)indent a row
  • <C-e> - delete next word (create a mapping in vimrc)

It's annoying when you delete something and overwrite your yanked text.

Use numbered registers! "0 to "9

  • "0 contains the most recent yank.
  • "1 contains the most recent deleted text
  • "0p - paste the most recent yank, even if you deleted something after yanking it

Chezmoi is a great tool for managing dotfiles. This is a shortcut to update the source state based on local changes.

chezmoi status | cut -c 4- | xargs -I % -p sh -c 'chezmoi add ~/%'

Github

“It enhances your memory and makes you more creative. It makes you look more attractive. It keeps you slim and lowers food cravings. It protects you from cancer and dementia. It wards off colds and the flu. It lowers your risk of heart attacks and stroke, not to mention diabetes. You’ll even feel happier, less depressed, and less anxious.”

Why We Sleep by Dr. Matt Walker

From an Interview:

I don't want to claim that programming is an art, because it really is mostly just about 'good engineering'. I'm a big believer in Thomas Edison's 'one percent inspiration and ninety-nine percent perspiration' mantra. It's almost all about the little details and the everyday grunt-work.

But there is that occasional 'inspiration' part, that 'good taste' thing that is about more than just solving some problem - solving it cleanly and nicely and yes, even beautifully.

A RPC is when an executable causes a procedure (subroutine) to execute on another computer, It's coded as if it were a normal (local) subroutine. You don't explicitly code the details for the remote interaction. You write the same code whether the subroutine is local or remote.

lsblk is a command to get info about connected devices.

Used when attaching drives.

Make aliased files the real file

for f in $(find . -type l -maxdepth 1);
    do cp --remove-destination $(readlink $f) $f;
done;

Save (write) a (read only) Vim file with sudo when you opened it without sudo-ing:

:w !sudo tee % > /dev/null

Broot is a tool that shows the contents of a directory on one screen, even its got lots of files or sub-directories.

I should see if I can use GitHub actions to generate html from markdown and run some shell and python scripts.

I was running a binary in Debian that was complaining about an environment variable not existing. I moved the binary into a $PATH directory and logged in as a sudo user. Why did this solve the problem?

As a hacker, or creator, or whatever the best label is, I always want to create something (usually code) and have it finished.

But a strange creativity and productivity boost comes from dabbling, dipping in and out.

I think that if the technical challenges aren't too hard, then the main criteria for success is creativity.

Creativity needs time away from the project, and sleep, to bubble up and let ideas grow.

Ultimately, the most successful path is usually the most interesting, because success has more consequences than failure. "Interesting" requires elements of novelty and surprise, and without creativity these elements can't flourish. 3/n

Dabbling results in more creativity than 6+ hours of strenuous work, and is more likely to give you satisfying results.

bar << foo bar will stop reading input when it reached "foo".

bar <<< "foo" foo is all the input. bar wont run interactively.

bar < <(foo:list) process subscription. Kind of like piping in the output of multiple commands.

Stack Overflow

I tried to read and write the same file in a pipeline, and got caught out by a race condition (why is the file empty?!). Do this instead:

some_script < file > smscrpt.$$ \
&& mv smscrpt.$$ file || rm smscrpt.$$

|| removes the temporary file if it errors.

$$ is the process ID and ensures that you always have a unique temporary file name.

jj - A stream editor
jq - A json processor
python -m json.tool

I like jq for pretty printing JSON output, jj for making JSON pretty or condensed.

This was really useful when optimizing the search index for this blog.

docker run -d ...
docker logs -f <ID>
docker run -it ...
docker run -itd
docker container attach <ID>
<C-p><C-q> -> detach from container interactively

stack overflow

Split long strings (or command outputs) onto multiple lines Find and replace a particular char (maybe :) with a \n.

... | tr ':' '\n'

... | sed 's/:/\n/g'
  1. Without noticing, create a .gitignore file with a single * in it.
  2. Spend a day trying to understand why ripgrep has stopped working for only 1 project.
  3. 😭 😭 😭

If you're cat-ing a file and the bash prompt doesn't start on a new line (cos the file you displayed using cat doesn't end with a new line char) the following will fix it:

cat <filename> ; echo

Cloning large repos, or repos with large files in them, doesn't work with git clone ... you need to use git lfs clone ...

So why is git lfs clone deprecated? What's the replacement?

Use nohup to keep a curl process running even when the terminal (tty?) session autocloses at 3am.

You can group shells into groups:

  1. ksh - korn shell and zshell
  2. sh - bourne shell and bash (the bourne again shell)

Because

  • zsh isn't a superset of bash.
  • bash is a superset of the bourne shell.