Making my current job slightly more on the command line
I have been playing around with the idea of creating a work interface of some sorts for various purposes. My job consists of doing a lot of database look-ups, checking different websites, juggling tabs, and overall mostly just web navigation and data managing.
But I thought I might take a stab at creating a kind of REPL environment for my job, so here we go.
The first big step is - how do I use this CLI? What's my user interface?
Recently I made the switch over to a Gnome environment, so I have been using fairly stock software with no real frills. But if I was asked to do a database lookup by a specific identifier, I have to go to my browser, load the web page of our in-house tools, then punch in the identifier, then navigate several pages to find the details I most likely need.
This isn't ideal for me anymore. I want to punch in a string and be taken to where I want to be, and to do this, I need some form of quick URL navigation combined with a fast way of opening web pages, most likely by invoking the browser directly or using
xdg-open. I have a bad relationship with
xdg-open, so I am simply going to pass arguments to the browser binary instead.
I need a way of accessing a REPL text environment quickly, so I installed Guake. Guake is a cool little terminal similar to the days of old where games had consoles you can drop into and do cool tricks with. Guake has a fast key shortcut to get to a terminal, so I gave this a whirl. Guake starts on login (if you set it), and you can toggle the terminal's visibility with
guake-toggle. Then you customize it to your heart's content.
This is where I wanted my programs to be invoked from, almost as if I were playing Quake/Doom again.
Now all that's left is to design a REPL environment that works, so.... Let's use Racket
Racket, when you start it from the command line, drops you into a REPL automatically. The next step would be to then find a way to create a program that doesn't get interpreted as if it were a program, but instead evaluates and lets you use the bindings in a REPL shell.
In order to do this, we must pass Racket some information, but we can create a REPL program with this command.
$ racket -l racket/base -if WorkCLI.rkt
The three arguments here are
-l, meaning library load a language. We want to use the
racket/base language as it loads the bare minimum Racket environment, cutting down on memory overhead. The other two flags joined into one are
-f, one meaning to start an interpreter environment, and the other is to load a specific file.
When doing this, in our
WorkCLI.rkt program, we must not define a language at the top level - that job is handled by the
-l racket/base part. If we define a language at the top level, the function is entirely different - it gets treated as a module space with a language definition instead, which we do not want. Instead, we simply declare the language with the
-l flag, then we import definitions using the
(You can also choose to not use the
-l flag, but it will default to the basic
racket language module, which imports a lot more than the
racket/base module, causing startup times to be slower and will use more memory.)
Here we can start creating definitions and actual Racket code and injecting it into the Racket REPL. So let's start writing.
Imagine we're in Python - we want to open a web page, how do we do it?
import webbrowser webbrowser.open("https://gitlab.com/")
That's pretty convenient! How does it work, magical Python bindings?
Er... No, not really. But it is pretty widely compatible with multiple platforms, it just uses common grounds between all browsers. Since we're on Linux, our desktop refers to the XDG media query system for opening certain files or paths, and if it sees a URL looking item, it defers it to some web browser on the system.
Unfortunately - we are not using Python. We are in Racket. So we must do this ourselves. If we want to make it cross-platform, we best also check for multiple browsers as well.
So let's say I started off with my naive approach - I had a few browsers in mind, but as the browser list expanded, so did my code...
; shortcut name (define fep find-executable-path) (define *browser* (let ([ff (fep "firefox")]) (if (not (false? ff)) ff (let ([chrome1 (fep "chrome")]) (if (not (false? chrome1)) chrome1 (let ([chrome2 (fep "chromium")]) (if (not (false? chrome2)) chrome2 (let ([edge (fep "MicrosoftEdge")]) (if (not (false? edge)) edge (error "Where is your web browser?!??"))))))))))
On another kind of day, I'd let this sit, because I think it looks downright hilarious and represents the grind of dealing with browser executable location. There are many other browsers in the world, but I'll focus on these four for now. Sorry IceCat, Konqueror, SeaMonkey, Lynx, Wget, Nyxt, Emacs, and all the other homies.
I decided to re-write this anyway since it resembles a very basic pattern anyway.
(define (find-bowser-pls listof-bowsers) (define (iter xs acc) (if (not (false? acc)) acc (if (empty? xs) (error "Where is your browser?!?") (iter (cdr xs) (find-executable-path (car xs)))))) (iter listof-bowsers #f)) ; Top-level browser definition (define *browser* (find-bowser-pls `("firefox" "chrome" "chromium" "MicrosoftEdge")))
Now you can support all browsers much easier without adding several
if scopes. Cool, right?
Now we can use a subprocess to open up our target browser with a URL each time we want to quickly access a web page with some primary key search or whatever you can rig it to. Let's write the Racket version of
; webbrowser.open/1 equivalent (define (webpage-open URL) (define-values (s i o e) (subprocess #f #f 'stdout *browser* URL)) (subprocess-wait s))
A subprocess is just a way of invoking other programs on the host system, and linking the main Racket process as the parent. The parent can run, monitor, suspend and kill child tasks all with relative ease, and that goes for browser programs too like Firefox.
Firefox isn't like other programs, it's more like a running server, so invoking
firefox under Racket doesn't always mean the whole Firefox program is going to live as a child of the Racket process. Rather, Firefox starts a daemon process and receives inputs instead. This is similar to how you can run Emacs as a server, so Emacs never dies as a process, but it can host multiple clients at once.
So by directly calling the browser binary with an argument input, it does indeed open up our web page. Next up is simple string formatting and creating easy and simple to use functions that do what we need.
The primitive way of creating a REPL environment is to simply define functions no different than what we're used to. If I want to search a term on Amazon, I simply string format the search term into a URL with a pattern variable.
; add on amazon (define (amzn-search identifier) (webpage-open (format "https://.amazon.com/s?k=~a" identifier)))
This is decent for one-off instances, but if endpoints or URLs ever change, we will unfortunately have to stroll back through some code and figure out what changed where and how to fix. Maybe not so bad in the long run, especially if it saves you a lot of time before changes occur.
If we were to wrap this around some kind of API-like web system, where the functions are pieced together like "webpage.com/catalog/add-product", we can encapsulate it using a bit better using constructive functions that wrap around URLs to create easy wrappers.
; build a functione dependent on *url* to be set (define (builder func-path) (lambda (identifier) (format (format "~a~a" *url* func-path) identifier))) ; create a catalog browsing shortcut (define catalog-browse (builder "/catalog/browser?ID=~a")) ; call it now (catalog-browse "01118778735")
Using a double-format call allows us to replace a variable in a template string with one that contains another variable. This allows you to chain templates together for further text replacement.
You can build up the logic however you wish. Whether it be with clever function currying like the above, but the unfortunate part is how we deal with the next issue - telling the user what commands are actually available in the REPL.
Normally when you're in some kind of REPL-like environment, or playing a text-based adventure game in the year 2022 and onwards, you normally rely on a function that spits out all possible commands you are legally allowed to enter into it. Racket does not provide this, but, unironically, Python does it a lot better. Using the
dir() function in Python, you can see almost everything in the runtime.
>>> dir() ['__annotations__', '__builtins__', '__doc__', ...] >>> dir(__builtins__) ['ArithmeticError', 'AssertionError', 'AttributeError', ...]
This is actually how I discovered some crazy things Python has, like the built-in
Ellipsis type, which is almost the same size as
None. Why it exists? Honestly, I don't really know...
>>> x = Ellipsis >>> print(x) Ellipsis >>> x.__sizeof__() 16 >>> y = None >>> y.__sizeof__() 16 # hmmm 🤔
But, all that aside, the
dir() function is very cool and lets you explore modules all from inside the Python runtime. Sadly, I don't think this exists in Racket. That's not a bad thing, but ... if it existed, I just don't know about it. If there was a way to expose the bindings defined in the namespace, maybe? I'm still not sure.
We need a help function for the user, in case they forget our weird functions. Or in most cases, if I were to forget. We can use a
hash type to associate function names with help messages of some sort, but unlike Python, we don't have help strings available. Instead we must do this afterwards.
; creating a help me screen of info (define *helplist* (make-parameter (make-immutable-hash))) (define (update-help k v) (unless (hash-has-key? (*helplist*) k) (*helplist* (hash-set (*helplist*) k v)))) (define (some-function x) (displayln "Done")) (update-help 'some-function "Does nothing and prints done")
Now we have a parameterized hash which relates names to help messages. But it's kind of annoying that for each function we would have to update the hash manually like that, but none the less, we can print out our help screen with:
; print out the help hash (define (help) (hash-for-each (*helplist*) (lambda (k v) (displayln (format "~a - ~a" k v)))))
I kind of admire the docstring format of Python a little bit more, a way of nudging a small string into the function declaration, so that way coders who dive into your code don't have to feel so alone. Granted I'm the only one using this, but those docstrings could be repurposed to match the help screen hash needs.
Problem is, the
define syntax doesn't fit this solution, so we might need to think of something else..... Or, we could use a macro?
Let's say for goodness sake that instead of using a macro, we just went with the normal
define syntax. I don't need to do the
update-hash function outside of the function, I can do it from within the function entirely, and that still works.
; docstrings, but as a function (define (do-nothing) (update-hash 'do-nothing "Does nothing, don't use") (displayln "Done")) > (help) do-nothing - Does nothing, don't use
Completely plausible to do this, but we don't want to register the hash each time we call the function, we only want to register it once. So even though
update-hash checks if the function has already been registered, we still don't need to make the call at all. Not necessarily a performance bump, but still not needed.
The other issue is that we have to re-declare the title of the function so
update-hash knows the name. Doesn't this seem kind of silly? We're inside the function, do we really need to spell it out?
I wrote a little macro for this that we could make everyone happy, and get to practice writing macros again. Let's take a look.
; The Action! macro, join help messages with code (define-syntax (Action! stx) (syntax-case stx () [(_ nameof helpmsg code) (if (not (list? (syntax->datum #'nameof))) #`(begin (update-help (syntax->datum #'nameof) helpmsg) (define nameof code)) (with-syntax ([siglist (syntax->datum #'nameof)]) (begin (unless (andmap identifier? (syntax->list #'siglist)) (raise-syntax-error #f "Invalid identifiers given" stx)) (unless (string? (syntax->datum #'helpmsg)) (raise-syntax-error #f "Given helpmsg not a string" stx)) #`(begin (update-help (syntax->datum (car (syntax->list #'siglist))) helpmsg) (define nameof code)))))]))
This might be one of the least complicated macros you'll ever see, and it wasn't that hard to put together. But the concept is we want to register a hash with a symbol pointing to a help message, then we do the literal function definition right after. The complicated
unless part is where we do the hash update, and then after that, the
define line is pretty simple.
Using this looks pretty basic in action.
; defining an action, which has a help string (Action! (say-hello name) "Says hello to whatever name is fed" (displayln (format "Hello ~a!" name))) ; use it > (say-hello "Steven") Hello Steven!
But more importantly, it registers in the hash we defined earlier too.
> (help) say-hello - Says hello to whatever name is fed
Now all the functions defined with the
Action! macro are registered to the help hash. No frills added, but
Action! may make your life more annoying for your text editing plugin depending on how you want to format it.
However, note that we have an
if statement very early asking if the
nameof variable is a list or not. Why is that? Because we have a bit of a branch to check for when writing code.
The Racket macro compiler sees these two pieces of code as matching the macro rule.
(Action! single-var "A single variable" 0) (Action! (a-func x) "Do something" (displayln X))
nameof variables are completely different - one is a single symbol value, while the other is a list. So to return different results, we need to see if the
nameof type looks something like a list or not, thus resulting in us creating different code paths.
The former rule here allows us to implicitly create functions, while the latter explicitly creates functions. Kind of neat difference, and separates writing styles. I lean more towards implicit in some cases where I can, because it reduces code strain, but there's nothing wrong with explicit either.
(define (func-maker X) (lambda (Y) (+ X Y))) ; implicitly declare (Action! add5 "Adds 5 to whatever" (func-maker 5)) ; or explicitly (Action! (add6 X) "Adds 6 to whatever" (+ 6 X))
Now with that all the way, let's move onto the last bit.
Say I don't know anything, I don't even know
help is a function. I should be printing a welcome message that says "hey, use the
help function if you actually need help", but a few lines down and some forgotten days, maybe that's not clear unless you reboot the program.
Let's pretend in some world we just make a mistake, and we need help. How do we go about that? What happens if I type in a function wrong?
> (do-somethinG) ; accidental capital letter ; do-somethinG: undefined; ; cannot reference an identifier before its definition ; in module: top-level ; [,bt for context]
When we try to reference a binding, all we get are exceptions that tell us our issue literally, from the perspective of the runtime. Alone, it won't tell us that we were off by one letter, or we typo'd and put a capital letter in some place that didn't need it. There's no error detection like that to really assist the user.
But also, I don't plan on adding that feature either currently, because it would involve a lot of debugging I don't know if I want to support right now. Maybe in the future I'll think about how to handle this, but for now I want to be able to show the help message when an error occurs. How do we go about this?
My familiarity with errors, or exceptions, is simple - you can run code, but even if you account for all possible return paths and conditions, something can always break, even without you account for it. Out-of-memory, network disconnections, process gets killed by the admin, or your computer turns off. Exceptions are designed as fall-backs for when certain errors occur that might not necessarily be the fault of the user.
So how does one handle exceptions normally? By using
; handle some code gracefully with another function (with-handlers ([exn:fail? (lambda (e) (displayln "Error caught"))]) (error "I am raising an error!!")) ; output Error caught
We went from verbose error message with details, to a much more simple string output displaying the worst error information imaginable. But hey, this kind of works for us.
The problem is there is no way we can perform
with-handlers in a REPL environment, because there's no body for which we can bind this macro to. When the REPL runs, it's not like we can bind it inside
with-handlers. Instead, we need to defer to some special parameters that exist, namely
; Help the poor user (uncaught-exception-handler (λ (x) (begin (displayln "Invalid pattern or syntax") (displayln "Try one of the following commands") (help) ((error-escape-handler)))))
uncaught-exception-handler is a specialized parameter called when exceptions are raised in the Racket runtime, but don't have handlers to deal with them. This default parameter function is pretty commonly used, because it's not like there's better functions to handle it.
In our REPL environment, this is our one and only way of converting all sorts of errors, into something that can assist the user in doing things properly and getting what they need done. By overwriting the default handler, you can create fun REPL applications this way, you could even make games. It's still a Racket shell, so valid Racket code will always work, but I thought this was a fun thing to try and do.
So after some hours of experimenting, getting Guake to run properly on keybind, learning about how keybinds work in Wayland, and writing this entire program up, I now have a working CLI for job oriented purposes. 😄
I can pop down my Guake shell, pop in a command, and it will either direct me to the web page I need, or perform other various office-related duties for me. It's kind of cool, and I hope I can make some co-workers jealous with my Linux desktop.
The better part will be how I connect it to my other Racket programs, but I'll have to work on that part. Should make for some interesting developments.
Alternatives I have tried:
xfce4-appfinderor something to run the programs - creating multiple scripts linked to some random binaries or whatever didn't seem very ideal. I at first thought about using
rofito do what I'm doing now, but I figured Guake was more practical.
That is it for now, thank you for reading. 🦖