.. index:: Unicode;UTF-8

Character sets
==============

Text files are assumed to be in Unicode and encoded in
UTF-8. Generated list files are also encoded in UTF-8.


.. index::
   pair: characters;HP-41
.. _sec-hp41-unicode-chars:

HP-41 character set
-------------------


The HP-41 uses a character set that resembles ASCII with some
additional characters which cannot be represented in ASCII. When using
language constructs that are intended for the HP-41 alpha characters,
some Unicode characters are converted to the corresponding HP-41
character.  The following table lists those characters with the
corresponding HP-41 character code and Unicode code point in
hexadecimal.

.. table:: Unicode table
 :widths: 1 3 2 3
 :column-dividers: none single single single none

 +----------+---------+------------+----------------+
 | Symbol   | Name    | HP-41      | Unicode        |
 +==========+=========+============+================+
 | μ        |micro    |``0C``      |``03BC``        |
 +----------+---------+------------+----------------+
 | ⊀        |angle    |``0D``      |``2280``        |
 +----------+---------+------------+----------------+
 | ≠        |not-equal|``1D``      |``2260``        |
 +----------+---------+------------+----------------+
 | Σ        |sigma    |``7E``      |``03A3``        |
 +----------+---------+------------+----------------+
 | ├        |append   |``7F``      |``251C``        |
 +----------+---------+------------+----------------+


In addition to these, there are some extra characters that do not
have any resemblance in Unicode, most notably the hangman style
characters. They need to entered using numeric constants, either using
the ``.con`` directive or by using numeric escape sequences in
strings.


HP-41 display characters
------------------------

.. index:: characters;display, display;characters

The HP-41 display character set uses a different encoding. This
character set is used at the MCODE level when dealing with the display
directly.


Automatic conversion
^^^^^^^^^^^^^^^^^^^^

In certain situations you need to enter literal strings in the display
character set. Two assembler directives ``.messl`` and ``.name``
are provided for this. They take a string argument and convert the string
in a suitable way for their respective use case.

``.messl`` is intended to be used when calling the mainframe routine
``MESSL`` which sends inline literal text directly to the display.

``.name`` helps you entering the name of a MCODE instruction. In
addition to converting the string to display characters, the order of
the characters are also reversed.

Both ``.messl`` and ``.name`` also set a bit in the final character to
mark the end of the string.


Display character encoding
^^^^^^^^^^^^^^^^^^^^^^^^^^

Display characters are encoded in 9 bits which can be visualized as
``UPPNNNNNN``, where ``PP`` are punctuation bits between
characters. The ``U`` bit can be seen as the next higher bit from
``NNNNNN`` which gives access to additional characters, i.e. lower
case letters such as  a--e.


Additinal halfnut display characters
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. index:: HP-41;Halfnut, Halfnut

In the middle of the HP-41 production run, the hardware was changed to
make the HP-41 cheaper to produce and sell at a more competitive
price. A slightly different display with additional characters was
introduced at the same time. This display can be identified by its
rounded corners. These later machines are normally referred to as
*Halfnuts*.

While the HP-41 mainframe code was not changed to take advantage of
the new display, the additional characters can be used from
MCODE. However, before doing so, consider that your program will not
display properly on earlier HP-41 calculators.

The extension of the character set is done by taking full advantage of
the ``U`` bit described above. On the original display, only 16
additional characters are provided, while on a Halfnut 64 additional
characters exist, for a full set of lower case letters.


Named MCODE instructions
^^^^^^^^^^^^^^^^^^^^^^^^

Extended characters can be used in MCODE instruction names and they
will display properly on a Halfnut HP-41, i.e. in ``CAT 2``, when to
assigned to a key and in program steps. [#Named]_

However, trying to key such instruction name on an HP-41 will not
work, which makes it hard to actually use it. You may find some use
for this feature when constructing a ROM header, or perhaps for some
internal instructions not normally intended to be executed by the
user.


Text messages
^^^^^^^^^^^^^

Unfortunately, the mainframe routine ``MESSL`` which sends a string to
the display does not perform the same kind of re-arrangement as is
done when dealing with MCODE names. Since the string end is marked by
setting the bit above ``U``, they both end up in the same nibble
(``U`` is the ninth bit). As a result, a display character with the
``U`` bit set can only appear as the final character in a string
intended for ``MESSL``.

If you put such character at another position, your message string
will terminate early and the HP-41 will try to execute the rest of
your string as MCODE instructions, which is probably not what you
intended.

.. rubric:: Footnotes
.. [#Named]
   This works because the bits are arranged slightly different way
   compared to how they are normally encoded for display characters.
   As a side-effect, no punctuation characters can be used in MCODE
   instruction names.
