I’ve recently read David Allen’s Getting Things Done  [1], more commonly known as GTD. I’ve taken to the approach, especially the concept of the next action list and the six level model for categorizing work. I’ve even integrated  [2] my next action list into Obsidian, after years of avoiding the latter for task tracking.

One other tool that Allen swears by in GTD is the tickler file, or suspense file  [3]. It is a technique for ensuring that you are reminded of tasks/projects/notes at a particular time. You could implement this with physical files and folders (see Fig. 1 below for the GTD version), a To-do app that supports reminders, post-it notes, etc. I spend the majority of my time working in front of my computer, and a lot of in in Obsidian, so having my suspense file right there is valuable.

I’ve written a short DataviewJS implementation of a tickler file that I use in my Daily Note template. Below, I go over the additional metadata that I put in the frontmatter of my notes, show the JS implementation, and show how it gets used in my templates. You can skip ahead to the Gist with the implementation if you’d like!

Implementation

I wanted to be able to easily “move” the tickler file ahead in time, as well as do something you cannot do with a physical suspense: put it in multiple folders at the same time. The easiest way to achieve this is to add tickler and a tickled arrays in my frontmatter. The first is used to indicate in which folder the file (or note in this case) should be, and the latter to mark the note as reviewed at a particular date. This allows me to have mutliple reminders in the future.

---
tickler: [2025-09-11, 2026-03-01]
tickled: [2025-09-11]
---

In the example above, I set a reminder for September 11th, and I have marked it as reviewed (through the tickled field) on the same date. I would then expect this note to only appear in my March 1st, 2026 daily note. If tickled had been empty, I would expect the note to appear in my daily note template starting from September 11th onwards (i.e. before March 1st, 2026).

My initial, simpler, query did not produce the expected behaviour:

TABLE aliases as Title, file.etags as Tags, tickler as Date
WHERE tickler AND max(tickler) <= date(2025-10-04) AND max(tickled) < max(tickler)

The use of max broke the ability to mark notes to be tickled at multiple future dates, as this query would only show the later date. I had to switch to a simple DataviewJS implementation that allowed more complex logic around matching time periods. Here is the script in its entirety below and in this Gist.

// Match up time periods for the tickler file and show items that haven't been tickled yet.
// Match up time periods for the tickler file and show items that haven't been tickled yet.
//
// My tickler notes have two properties set up:
//   - tickler: [YYYY-MM-DD, YYYY-MM-DD,...]
//   - tickled: [YYYY-MM-DD, ...]
//
// I want a note to appear in my tickler file whenever the current date is larger than the largest element
// in tickler smaller than the largest element in tickled. That will allow me to have multiple tickler
// dates in the future.
// @param { dv.page }           page - page fed through a dv.page query.
// @param { dv.luxon.DateTime } date - date passed as as luxon.DateTime object,
//                                     the same as the DataView dates. Makes it easier to do
//                                     date comparisons.
function tickler_file(page, date) {
    // Helper function for comparing Date objects.
    function isSameOrBeforeDay(d1, d2) {
        return d1.hasSame(d2, "day") || d1 < d2;
    }

    debugger;

    if (!date.isLuxonDateTime) {
        throw new Error(`You passed an invalid date into the query.
            Make sure to wrap the date in dv.luxon.DateTime.fromISO('YYYY-MM-DD')`);
    }

    if (!page.tickler) {
        // No tickler field, probably a bad query. Simply return false.
        return false;
    }

    const tickler_dates = Array.isArray(page.tickler)
        ? page.tickler
        : [page.tickler];

    // If it hasn't been tickled yet, find the minimum date in the tickler array.
    if (!page.tickled) {
        // Find the min
        const min_tickler_date = tickler_dates.reduce((latest, current) =>
            latest < current ? latest : current,
        );
        return isSameOrBeforeDay(min_tickler_date, date);
    }

    // If it has been tickled in the past, check whether the next tickler date is the same or before
    // the current date.
    const tickled_dates = Array.isArray(page.tickled)
        ? page.tickled
        : [page.tickled];
    const latest_tickled = tickled_dates.reduce((a, b) => (b > a ? b : a));
    const remaining_ticklers = tickler_dates.filter((d) => d > latest_tickled);

    // If there are no remaining ticklers, we are done.
    if (remaining_ticklers.length === 0) return false;

    const next_tickler = remaining_ticklers.reduce((a, b) => (a <= b ? a : b));

    return isSameOrBeforeDay(next_tickler, date);
}

exports.tickler_file = tickler_file;

If tickled is not set, we just need to check the minimal value of tickler against the current date. If both fields are populated, we find the the latest date in tickled, and check whether there are any fields in tickler that are later than that. If there are, we check that against the current date. And that’s about the size of it.

Note that we reuse the Luxon library to compare dates here. It is already used by DataView to parse the fields of the tickler and tickled array. It is easier to do this on query side, because this function is being loaded in its own sandbox, and doesn’t have access to the same environment DataView does. The script checks whether the date passed it is a Luxon.DateTime object.

This function gets called in my Daily note template as:

var utils = await dv.io.load("/Utils/dataviewjs_utils.js", "text");
eval(utils);
let pages = dv.pages()
	.where(p => p.tickler)
	.where(p => exports.tickler_file(
		p,
		dv.luxon.DateTime.fromISO("<% tp.date.now("YYYY-MM-DD", 0, tp.file.title, "YYYY-MM-DD-dddd") %>"))
	);
dv.table(
	["File", "Tags", "Date", "Date tickled"],
	pages.map(p => [p.file.link, p.file.etags, p.tickler, p.tickled])
);

I am using Templater to name my daily notes as “YYYY-MM-DD-dddd”, and so the tp.date.now call above just parses out the date from the file name.

Fig. 2 shows a rendered query in my Obsidian Vault. Once that blog post is published I can finally close that project and mark the file as reviewed in my suspense file!

Conclusion

Hopefully you’ve found the post useful, especially with the link to the Gist at the top. This post basically like reads like a recipe blog, except I didn’t refer to 9/11.

If you’d like to see my other posts, the easiest way to consume this site is through RSS. There are even feed categories for selected viewing: music, writing, technology. I love RSS, and I think you should  [4] too!

Resources

1. D. Allen, Getting Things Done: The Art of Stress-Free Productivity, Revised edition (Penguin Books, 2015).
2. AlanG, “GTD with Obsidian: A Ready-to-Go GTD System with Task Sequencing, Quick-add Template, Waiting-on, Someday/Maybe, and More”. Published on August 21, 2023. Accessed on September 16, 2025.
3. Cory Doctorow, “Keeping a Suspense File Gives You Superpowers”. Published on October 26, 2024. Accessed on April 21, 2025.
4. Cory Doctorow, “You Should Be Using an RSS Reader”. Published on October 16, 2024. Accessed on April 21, 2025.

I consume textual media from various sources: RSS feeds, Reddit, friends sharing links, etc. In conversations, I would often find myself saying “I read that somewhere, I can’t remember where”, and then try to turn to Google to find the article again, never to find it.

I also struggled to internalize the ideas, concepts, and conclusions from the media I consumed.

To try to fix this, I added some structure into how I read and process articles, blog posts, papers, and even podcasts, so that I could make better sense of them, and hopefully gain some deeper understanding of the subjects I was reading about.

After months of experimentation, I developed a process that works for me. It revolves around using Zotero and Obsidian together. By using standardized, day-based, nested collections in Zotero and an idempotent import template in the Zotero for Obsidian plugin, I can easily refer to past articles, and refer to them when writing long-form notes or articles in Obsidian.

Goals

This process aims to:

• record and organize papers, articles, blog posts, podcasts, and other media, with some metadata (when I read them, notes about them, etc), ideally with minimal effort;
• if possible, include a way to store snapshots of the media;
• integrate with my existing Obsidian vault.

Obsidian is a very popular Markdown editor, and I have been using it daily as a way to write down pretty much anything: journal entries, daily todoes, longer form notes about subjects I’m starting to learn, how to properly file my taxes… It has a rich plugin ecosystem that can change it from an editor to a full dashboard for your life.

Zotero

Zotero is the de facto tool for collecting and annotating academic articles. It can automatically add items based on ISBN, arXiv ID, DOI, and other well-known identifiers. The Zotero browser extension, called the Zotero Connector, also acts as a power web clipper so you can ingest blog posts and articles from the web. With version 7.0, Zotero introduced the ability to highlight and annotate locally saved web snapshots—an essential feature in my workflow.”

To organize my daily reading, I created a top-level collection called Daily, which is then organized in subcollections. See the tree

Figure 1: Screenshot of my Daily Zotero collections.

This structure acts both as a way to remove the friction of deciding which collection an article should go into, and to provide some structured metadata to help integration with my Obsidian vault. I always start by adding an article into its appropriate daily collection and, once I’ve read and understood the article a bit more, into a more topical collection, e.g. “Compiler Development”. As for the latter, collections are included in an easily parsable field using Dataview, making programmatic imports and querying into Obsidian easy.

Another Zotero option that I like to enable for this use case is the extensions.zotero.recursiveCollections option. As the name implies, it means that parent collections also show the items in their children collections. This makes it easier to do literature retrospectives, as I can easily look at a month’s worth of reading by choosing a month’s collection. The setting is only available through the Config Editor at Edit→Settings→Advanced→Config Editor. It’s a similar interface to Firefox’s about:config.

I considered and tried other options for tracking articles, like adding a read-on-date key in the Extra field in Zotero, or as a tag, but I always ended up either forgetting, or simply dropping an #inbox tag and then never looking at the article again. Moving the data collection at ingestion time made me do it more consistently. Let me know if you have a better solution!

Obsidian

On its own, Zotero offers a functional setup. You can easily access and search through a day or a month’s worth of articles, and the literature notes associated with them. Still, I wanted to explore the possibility of integrating this data into Obsidian, as it is where I draft most of my long-form notes and articles, and the idea of having all of my notes searchable from one location was pretty appealing.

The main enabler for this is the Obsidian-Zotero integration plugin for Obsidian. It allows you to import citations, notes, and even PDF annotations from Zotero into Obsidian. My workflow here is a modified version of Alexandra Phelan’s, described here and here.

The Zotero integration plugin allows you to import items from your Zotero collections following a user-defined template. The Zotero item is imported as Markdown file with arbitrary metadata. The templating system is based on Mozilla’s Nunjucks system. It is a decent templating system, but so far it does feel as powerful as Jinja.

One of the goals of importing my notes from Zotero was idempotency and bulk imports. I wanted to be able to modify items in Zotero, and have the changes reflected back in Obsidian. (This is a one-way street, I do not and cannot modify Zotero items from Obsidian). To do so, I use the Zotero unique item id (shown as key in the image below) as the imported file name.

Figure 2: Settings used to allow for idempotency for bulk imports and re-imports from Zotero to Obsidian.

Figure 3: File listing of literature notes imported from Zotero in my Obsidian vault.

This seems like it would limit search, but you can fix it by adding aliases in the import, e.g. the title of the item. That is a good segue to talk about the Nunjucks template that I use.

The Nunjucks templating system is very flexible. It allows for looping constructs and some conditional logic. My own import template is pretty long, with approximately 90 lines, so I’ll break it down here. You can see the full template here.

Let’s start by looking at the YAML frontmatter. There’s a lot in there, but for me the most important fields are title, year, aliases, and collections. Both the title and aliases are indexed by Obsidian search (and Omnisearch), so when you want to link to a Zotero import, starting to type [[]] will give you suggestions based on the title or citekey, so you can easily search by author or title. I use the collections field for a specific DataviewJS query that allows me to see what articles I’ve read on a particular day in my daily note, but this is a topic for another day.

---
title: '{{title | replace("'", "\"")}}'
year: {{date | format("YYYY")}}
citekey: "@{{citekey}}"
{% if itemType -%}
itemType: {{itemType}}
{% endif -%}
{% if itemType == "journalArticle" -%}
journal: {{publicationTitle}}
{% endif -%}
{% if itemType == "podcast" -%}
podcastSeries: "{{seriesTitle | lower}}"
{% endif -%}
{% if volume -%}
volume: {{volume}}
{% endif -%}
{% if issue -%}
issue: {{issue}}
{% endif -%}
{% if itemType == "bookSection" -%}
book: {{bookTitle}}
{% endif -%}
{% if publisher -%}
publisher: {{publisher}}
{% endif -%}
{% if place -%}
location: {{place}}
{% endif -%}
{% if pages -%}
pages**: {{pages}}
{% endif -%}
{% if DOI -%}
doi: {{DOI}}
{% endif -%}
{% if ISBN -%}
isbn: {{ISBN}}
{% endif -%}
aliases:
  - "{{citekey}}"
  - '{{title | replace("'", "\"")}}'
dates_edited: []
{% if collections -%}
collection:
{% for collection in collections | filterby("fullPath", "startswith", "Daily") -%}
  - {{collection.fullPath}}
{% endfor -%}
{% endif -%}
---

As for the note content itself, there are a few things that I wanted to be able to do:

• search through the abstract of the item;
• search through my notes;
• see which collections an item belonged to;
• import Zotero tags into Obsidian tags;
• navigate to the item in Zotero from Obsidian;
• use the graph view to see related articles;

Let’s go through the content block by block. There are a lot of brackets, and escaping of quotes, so brace yourself (pun intended).

## [{{title}}]({{desktopURI}})

{% if hashTags -%}
{{hashTags | replace(",", " ")}}
{% endif %}
### Attachments

{% if attachments %}
{% for attachment in attachments | filterby("path", "endswith", ".pdf") -%}
  - [{{attachment.title}}]({{attachment.desktopURI}})
{% endfor -%}
{%- endif %}

The first heading is a link that will open up Zotero at the particular item. This will render to something like this:

[Never Split the Difference: Negotiating As If Your Life Depended on It](zotero://select/library/items/2QI3X6TM)

a type of link you’re used to seeing if you use the Zutilo plugin.

The two next blocks import the tags, and ensure that they are formatted in a way that Obsidian understands (not comma separated, but space separated).

## Reference

{{-bibliography.slice(2,-1)}}

## Collections

{%- for collection in collections %}

  - [{{collection.fullPath}}](zotero://select/library/collections/{{collection.key}})

{%- endfor %}

The Reference block only shows the rendered citation based on your citation style. Honestly I don’t use it that much, but it’s a nice to have. The other block shows which collections the item is a part of, and allows you to navigate to the collection in Zotero by clicking the link.

## Notes

{% for note in notes -%}
### [Note {{loop.index}}]({{note.desktopURI}})

{{note.note}}

{%- endfor %}

## Abstract

{{abstractNote}}

## Related notes

{% for relation in relations -%}
  - [{{relation.title}}]({{relation.itemKey}})
{%- endfor %}

I then import all of my notes as-is in Obsidian, and provide a Zotero link to the note itself. The abstract is a simple block of text to import. I also import the related items in Obsidian, and that builds up the local graph view in Obsidian, increasing discoverability of notes.

Conclusion

Overall, I am satisfied with this workflow, even though there is more overhead than I would like to. Adding blog posts and web pages to Zotero can take a bit of time, as not all sites publish proper metadata, and I need to manually tweak the items. For some types of media, like podcasts, writing a Zotero translator (like this one for LWN) would probably save me some time.

Appendix: Essential Tools and Plugins

• Zotero;
  • Better BibTeX plugin (to populate the `` field);
  • Zotero browser extension;
  • extensions.zotero.recursiveCollections to see nested collections’ items in the parent collections;
• Obsidian;
  • Zotero Integration plugin;
  • Dataview (optional, but very convenient to query literature notes).

Numen is pretty much what you would expect out of Grossmann, Morean, Münzner, and Klausenitzer: highly technical compositions that you can somehow grasp on the first listen, and deepen your appreciate for over time. The atmosphere is very similar to their earlier albums, The Malkuth Grimoire and Liquid Anatomy, with riffs straight lifted straight out of Dyson Sphere, and will feel familiar to fans.

This is my main issue with the album, as it feels like the theme, and especially the lyrics, are starting to feel stale. I do enjoy Cthulu-esque monsters of the dark, squids, and unspeakable horrors hiding the eternal darkness of the cosmos, but I can already listen to The Malkuth Grimoire anytime. The Cambrian Explosion, which is a very fun track and packed to the brim with demonstrations of skill from all performers, seems like a retelling of Liquid Anatomy’s Rise of the Cephalopods, but with late Ayreon level of cheesy lyrics. Specifically, I want to call out

In the muck
Things start to fuck

which at least one reviewer seemed to like, but I just can’t get over.

Speaking of that particular review, I must say I disagree with their assessment of the titular song, Numen. While it exhibits the same issues of lifting a lot of content from The Malkuth Grimoire and rehashing the same story and themes, it is insanely catchy. It has the right mix of clean and harsh vocals, and right amount of tension and release. I particularly like the transition between Numen and Recursion. It feels almost like Super Mario 64’s endless staircase, but unfortunately no Shepard tone was used.

In short, this is definitely an album that will stay on list of go-tos, like Alkaloid’s two other albums. I wish the band finds new sources of inspiration for their writing, both musical and lyrical, and that they can find a way to renew their themes. And now begins the long wait for the fourth Alkaloid album :)

So, in no particular order, and not even my going by my most listened albums this year, here are some thoughts about the albums which left good impressions on me in 2021.

The Future Bites Steven Wilson Released January 29th, 2021 YouTube Music | Spotify

Wilson, who brought us, in my opinion, the definitive versions of Deliverance & Damnation, has been productive in his solo musical career as well. The overall effort that is The Futures Bites definitely rivals Hand. Cannot. Erase., and the insane banger Personal Shopper is definitely the song of the year. It is instantly catchy, yet interesting enough to be worth a second, third, or 50th listen.

The Future Bites sees Wilson embrace his love for electronic music in his solo albums (he’s definitely explored this in other projects). While he had previously explored mixing progressive and electronic elements within a single song, Personal Shopper is a fully electronic banger.

Vertiges Gaspard Released January 29th, 2021 Bandcamp

I might show some slight local bias here, but Gaspard is one my favourite discoveries of the year. The band describes themselves as being driven by the musical exploration of musical landscapes, and this album (as well as their previous one) delivers on this premise. The sometimes atmospheric and sometimes quite heavy riffs of the album are well balanced, and play off each other very well.

I also recommend their first release: La Forêt de Gaspard, of which Vestiges is pretty much an evolution. It is a study in textures and atmosphere.

The Stuff of Legends Challenger Deep Released October 29th, 2021 YouTube Music | Spotify

While the previous entry on this list is a slow exploration of a supernatural landscape, The Stuff of Legends is compact and energetic half-hour of satisfying math rock riffs with some great solos. This fast-paced album captures your attention for its entirety.

Their music video for The Great Calamity is also pretty fun.

Swagger & Stroll Down the Rabbit Hole Diablo Swing Orchestra Released November 2nd, 2021 YouTube Music | Spotify

I hadn’t yet followed Alice down the hole of avant-garde metal, but this album by Diablo Swing Orchestra made me fall hard. The eclectic mix of prog, heavy metal, folk, and other genres, could make for a disjointed listen, but it is so well done that is actually just refreshing, and keeps you on your toes the whole album. Speed Dating an Arsonist is my favourite track of the album for its catchy bits, funny lyrics, and satisfying finish, but the entire album is worth a listen.

This is my first DSO album, and I definitely plan on going through their earlier work in 2022.

Fear of an Obtuse Earth Home Brewed Universe Released January 25th, 2021 YouTube Music | Spotify

Home Brewed Universe is the one-man band backed by Arka Sengupta from Hyderabad, India. Fear of an Obtuse Earth was an early coup de coeur for me. It came out on January 25th, just before Vertiges. I was immediately enthralled by the rough metal sound, and the interesting prog melodies.

It suffers from a bit of repetition towards the end though. Home Brewed Universe also release Tranquil Taboo Time this year, which feels a bit more of the same, and suffers from the same repetition issue.

Sorella Minore Teramaze Released May 11th, 2021 YouTube Music | Spotify

Like the entry above, Australian band Teramaze released two full-length albums this year. While Sorella Minore unfortunately suffers from the 2112 syndrome, it also has its redeeming quality in that the the bombastic first track is makes it an excellent album. At once both easily digested and surviving more attentive listens, the 25-minute epic manages to make you forget its length by carefully layering complex compositions as time goes on.

This album is a continuation of the story set forth in Her Halo, but I must admit I haven’t paid too much attention to the lyrics. Even skipping the story element, the four vocalists put on an amazing performance.

Days Before the World Wept The Agonist Released October 15th, 2021 YouTube Music | Spotify

It is always good to discover a band with a female vocalist who can do both clean and harsh vocals, especially when it is a local band! Vicky Psarakis, who’s been with the band for their last 4 releases now, is heavily showcased through the album, and to the benefit of the album.

Dark Horse White Horse Dark Horse White Horse Released April 16th, 2021 YouTube Music | Spotify

Dark Horse White Horse is a collaboration between Marcela Bovio, Ruben Wijga, and Jord Otto, the last two both ex-ReVamp. This heavy album is a 22-minute compact EP that showcases each member’s talent. Mixed by Joost van den Broek, this record is definitely heavier than what I’m used to hearing from Bovio, who I have mostly heard through the lens of Arjen Lucassen’s rock operatic universe, and it works quite well with her vocal style.

To Where the Light Retreats Hannes Grossman Released June 1st, 2021 YouTube Music | Spotify

A very good album to scratch that Alkaloid itch. The Great Designer, the album’s opening song, could be a continuation of Rise of the Cephalopods, as they somewhat similar in composition and theme. I pretty much like anything Hannes Grossman produces, so any negative comment should be taken as being relative to his other work. While I found Alkaloid’s albums to be stonger overall, I very much enjoyed To Where the Light Retreats. It has the characteristic heaviness and technical prowess of his other work, but manages to be distinct while not feeling too incremental.

Senjustsu Iron Maiden Released September 3rd, 2021 YouTube Music | Spotify

It is impressive how Iron Maiden has consistently put out high quality records throughout their entire career (bar a few exceptions). While the titular opener is strong, my favourite part of the album is definitely the three closers: Death of the Celts, The Parchment, Hell on Earth. It seems that Maiden recently started liking longer songs again (all three songs have runtime upwards of 10 minutes), and I can only approve.

A View from the Top of the World Dream Theater Released October 22nd, 2021 YouTube Music | Spotify

DT’s outings in the last couple years have been hit-and-miss for me, but A View from the Top of the World sees them returning to their more traditional formula, and it is a definitive success. Like Maiden’s Senjutsu, this record boasts of a stronger opener and an amazing closing trio. The titular song, which is a 20-minute epic, doesn’t have the same impact as my nostalgia-coloured memory believes, say, Octavarium, to have, it is definitely a masterpiece of the genre. John Petrucci seems to have finally reached the speed that Mikael Åkerfeldt reckons he has.

Honourable Mentions

• Void – The Hollow Man Based on the T.S. Eliot poem of the same name. I love their interpretation of the poems, even if black metal is not my favourite genre.

• MFTJ – My Mom’s Getting a Horse Zappa-esque rhythms, quite quirky. The guitarist, Mike Keneally, was involved in the Zappa 1988 tour and is quite accomplished. Scott Schorr, the other half of the two-man band, has been involved in the prog scene as a producer for decades.

• Acrylazea – A Cavalcade of Cosmic Calamity An interesting blend of genres, with Unexpect influences. Similar to DSO. (Another avant-garde metal? I apparently have a new favourite genre.)

• Keor – Tearoom Clean riffs, and good musicianship. A good all-around album. It bills itself as emotionally explosive, and I think Keor delivers this with Tearoom. LEARNING GOD especially.

• Neptunian Maximalism – Solar Drone Ceremony Definitely something that needs to be listened to multiple times to be digested, studied, and understood. I found it a little too over the top to make the list. Angry Metal Guy recommends their earlier Éons however, so I’ll probably take some time to look into Neptunian Maximalism.

• Others By No One – Book II: Where Stories Come From Not sure if genius or insane.

• Prospectors – Proven Lands Another avant-garde album, definitely my favourite genre of the year. Or maybe I just enjoy experimentation and exploration in music. This debut album has an interesting mix of aggressive parts, but also slower build-ups leading to cathartic releases that I’ve come up to expect of prog-rock albums.

On x86_64 architectures, building gcc such that it can generate both 32- and 64-bit code is easy: just use the --enable-multilib flag at configuration time, and everything will follow smoothly (see Arch Linux’s PKGBUILD for gcc, the packaging is a bit more complicated, but the build itself is simple).

It is not as easy when targeting the MIPS architecture, specifically mips64-elf. By default, the 32-bit ABIs are not part of the multilib setup. gcc exposes the multilib setup that was used to compile a toolchain via the -print-multi-lib option. On your own x86_64 system, you should see something similar to this (assuming you have gcc-multilib on your system)

valandil ~ $ gcc -print-multi-lib
.;
32;@m32

where the second line shows the use of -m32 in the build options for the gcc toolchain itself, and the libraries are stored in the 32/ directory. This toolchain is thus comprised of two sets of GCC libraries, one compiled with default options, and one with -m32.

If you had installed a mips64-elf toolchain with --enable-multilib, by default you would get the following output

valandil ~ $ mips64-elf-gcc -print-multi-lib
.;
soft-float;@msoft-float
el;@EL
soft-float/el;@msoft-float@EL

In order to enable other ABIs, specify other build options for the GCC libraries in general, you need to know about Target makefile fragments.

In short, target Makefile fragments are the gcc/config/<target>/t* files in gcc’s source tree. When you specify a target at the configure step, it looks for those in order to generate the Makefile that will actually build the gcc libraries ¹. For a mips64-elf target, the file that is of interest to us is the gcc/config/mips/t-elf file. By default, it sets three Makefile variables

MULTILIB_OPTIONS = msoft-float EL/EB
MULTILIB_DIRNAMES = soft-float el eb
MULTILIB_MATCHES = EL=mel EB=meb msingle-float=m4650

which instructs the gcc build system to generate libraries for combinations of these three build flags. Flags that are separated by a space are combined, while incompatible flag are separated with a slash. It is this fragment that generated the mips64-elf-gcc -print-multi-lib output that we saw above.

So, if you want to generate libraries for other ABIs, you simply edit (or create a new one!) that file to what you need. In my case, I wanted to have libraries compiled with -mabi=32, at it is the ABI used by most N64 games. So, I edited my t-elf fragment to be

MULTILIB_OPTIONS = mabi=32 msoft-float EL/EB
MULTILIB_DIRNAMES = 32 soft-float el eb
MULTILIB_MATCHES = EL=mel EB=meb msingle-float=m4650

If you were to compile this toolchain, you would get the following -print-multib-lib output:

.;
32;@mabi=32
soft-float;@msoft-float
el;@EL
soft-float/el;@msoft-float@EL
32/soft-float;@mabi=32@msoft-float
32/el;@mabi=32@EL
32/soft-float/el;@mabi=32@msoft-float@EL

MULTILIB_DIRNAMES controls the name of the folder in which the libraries reside. Note that this corresponds to the information before the semi-colon in the -print-multilib-output, while the string after describes the build options, prepended with @. and MULTILIB_MATCHES provides the synonyms of some build options.

There are other options than those discussed here. Feel free to read the Makefile Fragment of the gcc docs to have the whole story, or look at an example of custom target in glank’s n64 repo, under config (as of July 9th, this is only in the n64-ultra branch).

They are given by

where we can verify that they are indeed orthogonal on the unit disk:

For a given value of \(n\in\{\{0\}\cup\mathbb{N}\}=\mathbb{N}_0\), the possible \(m\) values are in the range \(m\in\{-n,n\}\) and increase in increments of 2. To simplify the enumeration of Zernike polynomials, various linear index schemes have been devised. In short, each possible pair of \((n,m)\) is assigned to a monotonically increasing index that we will denote \(j\). In this post, we’ll go over some index schemes and write Python scripts that will allow us to map form the linear index \(j\) to the quantum pair \((n,m)\). First, however, it will be useful to investigate the structure of the pairs.

The Triangular Structure of the Quantum Pairs

By simply enumerating the quantum pairs and arranging them on a grid where rows indicate the value of \(n\) and columns the value of \(m\), their triangular structure becomes readily apparent.

Since each row contains one element more than the previous row, the number of elements contained in the first \(k\) rows is given by the \(k\)th triangular number \(T_k\), defined by

For index systems that are monotonic in \(n\), i.e. most of them, we can directly relate the quantum number \(n\) to the index of the smallest triangular number larger than \(j\). Fortunately, the index can easily be determined via the formula  [3]

The remainder \(r=j-T_k+n\) can then be used to determine what column corresponds to that particular \(j\) value. The various linear schemes use different rules to assign a column to the remainder.

Noll Indices

The rules to assign the Noll index is to have it monotonic in \(n\) and then assign in order of increasing \(|m|\), where odd indices correspond to negative values of \(m\) and even indices to positive values of \(m\). The correspondence between the Noll indices and the quantum pairs is the subject of OEIS sequence A176988  [4].

Since the Noll indices start at 1, the first quantum number can be written as one less than the triangular number index, \(n=k-1\).

The \(m\) value can be computed from the remainder \(r=j-T_k+n\). The latter can serve as an index into the list m = [k for k in range(-n,n+2,2)], as it guaranteed to lie in the range \([0,n-1]\). Because of the rules governing the assignment of signs, using the remainder directly as an index into this list is rather tedious. A simpler way is to create a list that generates all the possible values of \(|m|\) and sort them in ascending order. For \(n=3\), that list would be [1,1,3,3]. \(r\) can be used as an index in this list to determine the value of \(|m|\). The sign is then decided by the parity of \(j\) directly, as stated by the Noll rules.

Here’s the Python code that implements this (a C++ implementation is available as part of my Zernike library)

import numpy as np
def NollToQuantum(j):
    triangular_numbers_idx = np.array(np.ceil((1+np.sqrt(1+8*j))/2),dtype=int)-1
    triangular_numbers     = np.array(triangular_numbers_idx*(triangular_numbers_idx+1)/2).astype(int)
    n = triangular_numbers_idx - 1

    r     = j - triangular_numbers
    rpn   = r+n

    # -- Have to work value-per-value here.
    for i in range(n.size):
        m_vec = np.array([j for j in range(-n[i],n[i]+2,2)])
        m_vec = np.sort(np.abs(m_vec))
        m[i]  = (-1)**(j[i]%2)*m_vec[rpn[i]]

    return n, m

\(m\) could be computed directly via the one-liner

m = (-1)**j * ((n % 2) + 2 * np.array((r + ((n+1)%2))/2).astype(int))

but in practice I found the code above to be faster, even though it generated a larger bytecode than the one-liner version.

Let’s work through an example value. For instance, suppose that we have \(j=8\), then the formula above \(k=4\) and thus \(n=3\). The remainder is \(r=8-10+3=1\). From our list above, we have that \(|m|=1\). Since \(j\) is even, \(m\) is positive, and we have \(m=1\). And indeed, \((j=8)\mapsto(3,1)\).

Phasics Indices

The Phasics index system is monotonic in \(n\), as Noll’s is, and also monotonic in \(|m|\). However, this system elects to always assign positive values of \(m\) to lower linear indices, regardless of parity, as stated in the SID4 6.4 User Handbook, Chapter 6.

From these characteristics, we see that Noll and Phasics only differ on rows where \(T_n\) is even, as We can thus use the same basic function above, but add the single-line

m[i] *= (-1)**(triangular_numbers[i]%2+1)

at the end of the loop.

Sources

1. L. Thibos, R. A. Applegate, J. T. Schwiegerling, and R. Webb, "Standards for reporting the optical aberrations of eyes," in Vision Science and Its Applications (Optical Society of America, 2000), p. SuC1.
2. J. C. Wyant and K. Creath, "Basic Wavefront Aberration Theory for Optical Metrology," in Applied Optics and Optical Engineering, Volume XI, R. R. Shannon and J. C. Wyant, eds. (1992), Vol. XI.
3. J. D’Aurizio, "Largest Triangular Number less than a Given Natural Number," https://math.stackexchange.com/q/1417583", accessed Sep. 26th, 2018.
4. R. J. Mathar, "A176988: Triangle read by rows which contains Noll’s indices of Zernike polynomials in row n sorted along increasing index of the azimuthal quantum number," https://oeis.org/A176988, accessed Sep. 26th, 2018.

Appendix: Table of index values generated by the algorithm

Of course, one could write their own I/O for their C++ production codes, but most libraries have built-in I/O functions and it’s just a pain. It’s much easier to get Python to read the C++ format!

The Issue

C++ represents a given complex number \(z=a+ib\) as (a,b). When using genfromtxt to read a file containing an array of complex numbers, the obvious

import numpy as np
x = np.genfromtxt("file", dtype=complex)

yields NaNs for the real part and 0s for the imaginary part. To get Python to understand the format, an easy solution is to first parse the elements of the array as strings, then use a lambda function to effectively cast the strings as complex numbers. Here’s the code.

The Code

def LoadComplexData(file,**genfromtext_args):
    """
    Load complex data in the C++ format in numpy.
    """
    array_as_strings = np.genfromtext(file,dtype=str,**genfromtext_args)
    complex_parser = np.vectorize(lambda x: complex(*eval(x)))
    return complex_parser(array_as_strings)

First, we load the array of complex numbers as an array of strings in Python. Then, we use np.vectorize to define a callable that takes each of the array and applies the complex(*eval(x)) to it. eval() takes the string and evaluates it to a tuple of floats. The * operator unpacks the tuple such that we are calling complex(a,b) properly. This returns a complex number in Python format.

A solution based on a genfromtxt converter could be more elegant, but I wasn’t able to find a way to apply a converter to every column of the input file instead of a specific column.

You can try it for yourself with these files. This was tested with Numpy v1.14.5.

The setup is quite simple, but requires two terminal windows. In the first terminal, SSH into your remote machine with

user@local-machine $ ssh -Y <remote.machine.com> -L 5900:localhost:5900

and then,

remoteuser@remote-machine $  x0vncserver -display :0 -passwordfile ~/.vnc/passwd

Now, in the second terminal on your local machine, you can simply type

user@local-machine $ vncviewer localhost:5900

The -L flag in the SSH connection sets up port forwarding from the local machine to the remote machine. In short, all unbound traffic that goes through the specified port on the local machine is forward to the other port on the remote machine. Here we choose port 5900 on both machines as it is the default port for x0vncserver, but both can be changed freely.

Make sure to generate a password file for VNC by using vncpasswd and pass it to x0vncserver on the passwordfile flag. Here we choose the default.

The same method with generic ports would read

user@local-machine $ ssh -Y <remote.machine.com> -L <xxxx>:localhost:<yyyy>
remoteuser@remote-machine $  x0vncserver -rfbport <yyyy> -display :0 \
                                         -passwordfile ~/.vnc/passwd

user@local-machine $ vncviewer localhost:<xxxx>

Until next time, cheers!