text-mode personal knowledge system
Find a file
2023-08-01 23:03:20 +02:00
data_schema fix(wikidata): use overloading in wikidata editor 2023-06-14 16:55:32 +02:00
examples fix(example): more realistic data 2023-07-30 18:31:56 +02:00
scripts/import_joplin refactor: src/lib structure 2023-05-06 08:55:56 +02:00
src WIP: feat(indexer): basic interface 2023-08-01 23:03:20 +02:00
.gitignore refactor: include pdel serializer and wikidata editor 2022-12-11 10:04:53 +01:00
.rgignore fix(wikidata): use overloading in wikidata editor 2023-06-14 16:55:32 +02:00
Cargo.lock feat: wikidata property overloading + resolve reference 2023-05-25 09:59:39 +02:00
Cargo.toml WIP: feat(indexer): basic interface 2023-08-01 23:03:20 +02:00
README.md fix: README.md 2023-07-30 18:34:15 +02:00

Popequer

Popequer is a way to organize your digital life around a text-based, entry-based, personal knownledge management system.

Built to be flexible, one of the initial goal is to have a way to edit and contribute to wikidata.org using text syntax.

But the goal is to be able to use it later as a collaborative knowledge management system to eventually replace wikibase.

This project is currently at the very begining with only the parser for now.

The popequer project currently have a unique mono-repo.

Components

  • The popequer data entry language parser (PDEL)
    • Parse an entry to an intermediate Entry representation (IER)
  • The wikidata interface:
    • The wikidata to PDEL translator lib
    • The parsed PDEL to wikidata translator lib
    • The normal wikidata client
    • The wikidata CLI editor that use the libs
  • The Database/Indexer:
    • Directory crawler, markdown parser and indexer
    • Convertion from IER to a rich, validated and linked database
    • The query interpretor (that use an adaptation of the PDEL for the query language)
    • Indexer CLI
    • Query CLI
  • User Front-end
    • Calendar view
    • Events/History view (Activity view)
    • Adaptative view (for any kind of relational and general database)
  • Convertion utils from and to common formats (CSV, JSON, etc)
  • Import/Export interface for specific uses
    • CalDAV interface
    • CardDAV interface
    • YouTube interface
    • E-Mail interface (IMAP)
    • File system interface
    • Any kind of interface to foreign software to pull activity data
  • Popequer notebook day-to-day CLI helper

External components:

  • Android app
  • Desktop visualizer

Quick reference

To only run one test with stdout print not hidden:

cargo test parse::test_parse::test_parse_entry_claim -- --nocapture --exact

The PDEL Parser

We will introduce a new markup language specification or format The purpose is to ease and simplify the creation, edition and management of wikibase-like items This boils down to being able to quickly make claims using human usable identifier or key Instead of remembering for each wikibase property an Id, we introduce a mapping between the key and the property id We will also introduce simple syntaxic sugar to prevent redundancy in our syntax It's a simpler form of the notation3 syntax which aims to a readable RDF syntax We aims to keep as close as possible to the RDF models but with specificities of wikibase in mind To put it in a nutshell, we want to create a language to edit wikibase with the same ease and practicality of the wikibase web interface.

The popequer database and data model

Compatible with the wikibase, the data model is relatively simple and allow to easily model any kind of knowledge.

The database must be design to allow for graph-like query search. Like: what are the friends of my friend? We could use neo4j but it's not very embededd. We could study what makes neo4j interesting as a graph database.

Ideally I would like to use an embeded graph-database

All datetime are UTC only (at least internally).

TODO

  • have the indexer add a ID after first discovery of the items,
    • the requirement for that was to have the whole syntaxical tree available in the parse output of an entry
    • we now have that, we can now safely add in text the claim: id: E000001, that way we can identify each item later for update
  • most basic database
    • write: encode popequer bunch of objects into binary file
    • read: decode binaries and have a way to view them
    • only a index on ID field to quickly find later
    • no SQL
    • search function that take O(n)
    • format: CBOR, BinCode, Protobuff
  • advance db search
    • construct custom indexs (eg. index for geocoords

Definition of the MVP

There are two versions of the MVP

MVP for personal use with neovim

  • Rust? treesitter extension that use the same codebase would be great
  • Rust LSP server on a notebook
    • Implement LSP "GoTo" feature to resolve references
    • Implement LSP
    • Provide completion of properties AND items name
  • Neovim plugin to add all the useful stuff
  • Basic CLI util
    • launch an indexation and update of the DB
    • get the status

MVP for wikidata use with neovim

Wikidata editor

Getting started as a user

use the indexer

cargo run --bin indexer -- -s ./examples/notebook/ -i ./.tmp/notebook_internals/