Development notes
Organon
Known Issues 😳
- Putting
id
attributes on backlink elements don’t work consistently. For instance, plain list items are not wrapped by an element. Perhaps I should add aTarget
inline element to the AST instead, and rework the backlink preview functionality to be more similar to what it was before (without fancy buggy portals). - Portal stuff is terrible and should probably be ditched altogether. Transclusion can probably work via
walk.query
expansions. - Attachment figures do not get recognized as figures. Related to AST Annotations.
- Anchor links to the same file but to a part with a different ID do not work.
-
organon-follow-mode
gets confused with headings with the same title.
Editor integration
Robust page following
Observations:
- Since headings already have anchors, it’s easy to use AST annotations to reimplement the current following feature but without the issue when there are headings with the same title.
-
Element-wise following requires some thinking, but it should end with something more robust.
Idea: if the start position is stored in AST and
Sync heading folds
Custom metadata
Use ondim? pageExp
could be available inside it, could be used to store stuff like todo states!
Precise backlinks
Basically: heading-level, element-level and inline-level backlinks.
We already store all the necessary information to provide this.
Also: more general forms of backlinks. For instance, timestamps and citations should be considered links.
Just a note, only forward links are stored, and this really what we should store. Backlinks are then queried from the DB via the links-to
thing.
Metadata files
Generated pages
Pages generated by metadata queries.
Cli
Cache the rendering functions
CLI command to clear cache
Runs the generator once without writing output, clear what is not used.
Querying
If done right, querying can generalize a bunch of patterns you find in a static site. For instance, creating a list of posts with a given tag or creating a list of tags.
[0/1]
Syntax
[0/1]
Syntax
The basic idea here is using affiliated keywords, more specifically backend key/value pairs, and bind expansions providing the query results in the context of its children.
[0/1]
Keyword querying with :wrapped
[0/1]
Keyword querying with :wrapped
-
Parse org key-value pairs
Org-formatted queries
We could even have a pure Org preprocessor
syntax (note: in fact, the
expansion would happen during rendering inside Ondim
).
This would require more changes to org-hs
:
- Expansion syntax for objects (and perhaps elements), e.g. using
{{{macros}}}
for inlines (and something else for elements).
Creating tables:
#+attr_query: :from notes :where (in "games" tags)
| Name | Time Played | Length | Rating |
|---------------+-----------------------+------------------+------------------|
| {{{q:title}}} | {{{q:m:time-played}}} | {{{q:m:length}}} | {{{q:m:rating}}} |
Or a list:
#+attr_query: :from notes :where (in "games" tags) :where (or (in "moba" tags) (in "crpg" tags))
- {{{note:link}}}
In principle, the output of all examples above could be generated in Org
beforehand with lisp, by using src
blocks. But the advantage here is that they
are updated automatically every time the page renders.
\mathrm{\LaTeX} support
Parallel compilation
- Note taken on
This will likely be superseded by the parallel map expansion I plan to put on Organon per se.
The way things are set up now , the \LaTeX rendering is done during the document rendering (from Org AST to HTML, via Ondim). This has some advantages:
- There is no need to encode the output in the AST, which can be limiting — as a result, it is easy to customize how the figure is laid out in HTML via templates;
- It fits better in the
expansion \iff plugin
metaphor I’m aiming for Organon; - Fragments are rendered lazily, just before they are shown in the browser page;
- It is more general in that it could more easily be made to work with input types other than Org AST.
But doing so introduced a performance problem, namely that Ondim rendering is not easily paralellizable because the monad carries a state. The proper way to reimplement that is via what I would call a session expansion
.
Session expansions
The basic idea is: we have a organon:latex-session
expansion which binds organon:render-latex
inside.
The org-mode-hs
libraries
org-mode-hs
libraries
Known Issues 😳
- Inner links are not working! and they were working before!!
AST Annotations
-
Note taken on
Some days ago I had a go at this and gave up. Sadly I don’t remember the problem anymore, but in the end I was sure I would have to parametrize with two types, likeOrgDocument element object
, and I was sure a single type parameter for metadata would not work.Acho que o problema era porque o Walk não iria funcionar bem, seria impossÃvel fazer o Walk construir as anotações aos poucos na hora de resolver os links. Mas pensando agora seria perfeitamente viável, só usar a instância
Functor
pra colocar uma metadata vazia no lugar antes de andar.
Investigate the use of trees that grow (note: certainly overkill. A simple type parameter for metadata is for sure much better) to allow annotations in the AST. Use this to implement precise page following for Organon.
- https://hackage.haskell.org/package/barbies-2.0.3.0/docs/Barbies.html#g:3
Interaction with export
Let the export backend define custom ways of expanding the metadata. Those expansions should overwrite the default ones. Probably this should replace the current customization options for the backend.
Ondim
Why Ondim?
Advantages of having the template language built in the target language:
- No need for special editor support; make use of all the existing tools.
- Less prone to errors; In some cases, it is even impossible for the templates to result in invalid documents.
- Context-aware means it’s possible to handle contextual rules like text escaping automatically.
- Easier to understand visually compared a mix two languages.
- More modular: no need to write a parser or writer if there are already existing ones.
Known Issues 😳
- When using
withExpansions
to bindfoo.bar
, wherefoo
already exists in the environment, the deletion offoo
leaks outside.
Subtemplates hack
- Some profiling shows one of the mostly costly things when rendering pages is text replacement of attrSub. This is because it parses the same text over and over each time the template is run, countrary to the template itself which is already parsed. I could make the loading functions parse the text once and store the AST as another template that attrSub just calls during expansion.
-
Note to self: difficulties are
- How to get the text templates from a document in an elegant way. The current way is to run Ondim with errors disabled.
- How to store the text templates in a place where they can be unloaded along with the file, or if they should not be unloaded (perhaps a source of memory leaks)
- Text duplicated in memory: bad?
- Rendering Whiskers/LaTeX ASTs is very slow for odd reasons. Simpler AST?
Store things temporarily
Say we want an expansion to run inside a monad with extra state, or even other side effects. We want to do that locally, because this inner state does not make sense outside.
-
Note taken on
This is tricky because the state is parametrized by the inner monad and used in contravariant position. So there cannot be a
mapOndim :: Ondim tag m a -> Ondim tag n a
without two functionsright :: forall a. m a -> n a
andleft :: forall a. n a -> m a
. Without specifying the right semantics, what we are after does not even make sense. -
Note taken on
Not that tricky anymore! There was only contravariance in
Filter
and now I’ve just removed it. -
Note taken on
Well, I was wrong above. The mere presence of
StateT (F m)
is contravariant inm
, because it takes the previous state as input. I’ve spent the entire afternoon attempting to make it work, just to realize this. 😢So, it’s impossible/not well defined.