locale 44.1 (5.8.99)

- Fixed an ancient buffer overrun problem in the DateToStr() patch.
  This was most apparent with country definitions that had four digit
  year numbers in them, such as italia.country or portugal.country.
  Since the length of a day/date/time string, as passed to DateToStr()
  cannot be greater than 16 characters (including the terminating NUL),
  a date string such as generated for the month December (e.g.
  " 1-Dicembre-1999", which is exactly 17 characters in size, including
  the terminating NUL) would cause the buffer to overflow. This caused
  all kinds of trouble. I limited the size of the string generated. While
  this is not exactly a good fix for the problem (the last year digit will
  "drop off"), it avoids the buffer overrun.

- Added default cases for the RawDoFmt() patch (no longer throws Enforcer
  hits on invalid formatting commands) and the ARexx character conversion
  code.


locale 44.2 (5.8.99)

- Changed the built-in default locale for united_states to use four digit
  year numbers.


locale 44.3 (9.9.99)

- In DateToStr() the '%y' code would calculate the year number minus 1900
  to obtain the two digit year number. I changed this to use the year number
  mod 100, which should do it.


locale 44.4 (24.9.99)

- The library now maintains proper usage counters for all locales it loads.
  Previously, locale.library would keep any locale loaded in memory
  indefinitely, including the associated language driver. Now everything
  that remains is the sys/dos.catalog file which, since dos.library does not
  and cannot maintain a usage counter for its strings, has to remain in
  memory once loaded (some dos.library users may cache strings).

- The library no longer bumps its usage counter every time the default
  locale is changed. This happens only once now, when the operating system
  patches are planted.

- The built-in "english" locale now has its usage count set permanently to
  1, preventing it from ever getting removed.


locale 44.5 (24.9.99)

- The ParseDate() code now specifically allows "%y" year numbers to
  have more than two digits.


locale 44.6 (7.11.1999)

- The catalog file loader no longer refuses reading catalog version
  strings if they are longer than 99 characters. It now reads a
  maximum of 99 characters and tries to make the best of it.


- When planting its patches with SetFunction(), locale.library now
  brackets all calls with Forbid()..Permit().


locale 46.1 (12.3.2002)

- Replaced 32 with MAXLANGSIZE or MAXLOCSIZE where appropriate.

- Added name length checks.

- Renamed charset-related internal functions from "English" to
  "Latin1".

- If a language driver provides a GetCodeSet function and this function
  returns a non-zero codeset and the language driver does not provide
  all charset-related functions, a charset driver is loaded (e.g.
  Locale:CharSets/111.charset for ISO 8859-15 Latin9, the number is
  the IANA charset number, see
  http://www.iana.org/assignments/character-sets) and used for the
  charset-related functions not provided by the language driver if
  it provides replacement functions for the builtin Latin1 functions.

- Now tries to open diskfont.library V46+ (no complain if it fails)
  and sets diskfont.library V46+ default charset from default locale
  loc_CodeSet.

- Created 111.charset ISO-8859-15 Latin9 charset driver.

- Created ISO15 language drivers from the available sources (only had
  the OS3.1 language driver sources available). No danskISO15.language
  driver, sorry, the dansk.language driver has some strange builtin
  charset function replacements I dont understand yet...


locale 46.2 (20.3.2002)

- Now needs diskfont.library V46+.

- Now uses the MIME name instead of the IANA number for the charset driver
  in Locale:CharSets/, e.g. Locale:CharSets/ISO-8859-15.charset.

- Renamed 111.charset to ISO-8859-15.charset.

- Renamed ISO15 language drivers to ISO-8859-15.


locale 46.3 (22.3.2002)

- Translated latin1.asm to ISO-8859-1.c

- Fixed Collate1 table bugs from original "english.asm" driver.

- Fixed CharAttr table bug from original "english.asm" driver.

- Fixed alien memory reads and random results of IsXXXX functions
  and ConvToUpper/Lower functions for characters > 0xFF.


locale 46.4 (25.3.2002)

- Fixed ToUpper/ToLower/CharAttr table bugs for '�' and '�' from
  original "english.asm" driver.


locale 46.5 (26.3.2002)

- Added diskfont includes.


locale 46.6 (26.3.2002)

- Moved TA_CharSet to diskfont.h.

locale 46.7 (23.7.2017)

- For better compatibility with legacy workbench versions,
  this release no longer requires diskfont V46, but uses
  it if it is available.
- Minor improvements, uses now utility for the math, all const
  data is now in the code segment, avoiding relocation.

locale 46.8 (5.1.2018)

- Disabled support for character sets right now as 3.1 does not
  have them. We may reconsider this feature later.

locale 46.9 (19.8.2019)

- Very minor optimizations in the locale.library Format() function.

locale 46.10 (6.6.2020)

- Modified RawDoFmt patch to be compatible with exec 45.x (putCharProc may be NULL)


locale 46.11 (7.9.2020)

- Reduced the number of header files included for prototype/interface
  functions.

- Added wrappers for the various ASM, REG and __far keywords which match
  what's now in <clib/compiler-specific.h>.

- No functional changes... yet!


locale 46.12 (9.9.2020)

- Reset the code indentation, scrubbed the number of empty lines which separate
  functions and code, added a few more empty lines to make the code a bit
  more easy to read.

- Still no functional changes... yet!


locale 46.13 (9.9.2020)

- Lucky #13!

- Dialed back the exuberant code indentation, in particular for the
  nested switch/case statements.

- Added { } braces to code for which it renders the results more
  readable.

- Still no functional changes... yet!


locale 47.1 (9.9.2020)

- Rewrote the OpenCatalogA() function so that it minimizes the
  time it may hold onto the global LocaleBase lock. This no
  longer happens until a catalog matching the requirements is
  found to be already in memory, or until a catalog matching
  the specs has been read. In case it turns out that a different
  Process has already read that catalog first, then that will
  be used instead.

- Trying to load a catalog from disk now only makes two attempts
  to open it, and not three. If the catalog can be found in
  PROGDIR:Catalogs, then that's fine. If not, it will check
  LOCALE:Catalogs instead, and if that doesn't work, then there
  will be no second attempt to maybe find it in PROGDIR:Catalogs
  after all.


locale 47.2 (11.9.2020)

- Replaced short integer type variables with normal int type variables.

- OpenCatalog() no longer checks for every language if PROGDIR: is
  valid. Once is enough.

- Replaced implicit non-zero value checks with proper explicit tests,
  using the correct type.

- Replaced untyped 0 with the specific correct type, e.g. NULL
  or '\0'.

- Where strings are compared, the result is no longer tested
  against 0, but against a preprocessor macro by the name of SAME,
  to make it easier to spot what is being done.

- Broke up combined assignment + test (e.g. if(result = AllocVec(..)))
  into separate assignment and test where possible.

- Hard-coded data structure and table sizes ("magic numbers"), e.g.
  10 and 16, now use either a sizeof() of the type in question or a
  macro which produces the number of table entries.

- When setting the global "language" environment variable, the
  name of that language is now truncated before it is copied into
  the name buffer passed to SetVar() instead of after it has
  been copied. This avoids trashing memory or the Process stack
  for language file names longer than 22 characters.

- OpenLocale(), if called with a locale name, no longer hangs onto
  the global LocaleBase lock when loading the requested locale from
  disk. It's not necessary to do so.

- Replaced Pascal/Modula-2 style while-loops with their equivalent
  'C' for-loops for better readability.


locale 47.3 (12.9.2020)

- The loop tests in ISO-8859-1.c broke the library. Oops! Back to
  square one. We shamble along with changing the table index
  variables from bytes to integers.

- OpenCatalogA() did not process all the languages in the table
  because the code which does that is at the very bottom of the
  loop, so you can't safely use "continue" anywhere. Bummer :-(

- Figured out why InitNumFormat() only checks the first 10 entries of
  locale->el_Locale.loc_Grouping[]. The size comes from the
  locale preferences files, specifically the CountryPrefs section
  and its 'cp_Grouping' table.


locale 47.4 (12.9.2020)

- No longer assumes that the locale->loc_LanguageName will have a
  ".language" suffix. Since OpenLocale() does not require this,
  expecting that the last 9 characters of the locale name can be
  removed is not helpful. If the name happens to be shorter the
  stack would be trashed, or worse. A faulty locale prefs file
  could trigger this :-(

- Checked which function parameters are better declared as being
  constant. Adapted the .sfd file and prototypes and interface
  files accordingly.

- The number formatting helper table setup was subject to an
  off-by-one error and could have initialized a 17th entry out
  of 16...


locale 47.5 (13.9.2020)

- The string data which is part of a catalog file must contain at
  least one translated string to be useful. The code which preprocesses
  the data for the catalog string lookup function depends upon that.
  Up until now this requirement was never even tested.

- Preprocessing of the catalog file's translated string data now
  verifies that the size of the string data is not larger than what
  the buffer allocated for it would hold. If the size of a string
  is found to be invalid, preprocessing will be stopped immediately.
  This at least provides a chance to keep any valid translation
  strings which were valid.

- If the catalog string data is found to be unusable, the catalog
  data will be freed and the catalog rendered safe for the catalog
  lookup code.

- If the catalog version string is malformed or corrupted, trying
  to process it is now aborted as soon as possible.


locale 47.6 (14.9.2020)

- Small changes for (hopefully) better readability.


locale 47.7 (19.9.2020)

- Replaced the assembly language version of the GetCatalogStr
  function with plain 'C' code. Not at straightforward as the
  original version but the lookup loop itself is not so
  different.


locale 47.8 (19.9.2020)

- Replaced the catalog string initialization with code that
  uses a data structure to show how the process works. Pointer
  and integer arithmetic will only get you so far.

- Removed the assembly language GetCatalogStr implementation.


locale 47.9 (19.9.2020)

- Simplified both the LoadCatalog() and LoadLocale() implementations
  so that they no longer do things multiple times for cleanup work (in
  case of error). Also, the code indentation no longer grows with each
  if (..) clause, rendering the code a bit more readable.

- If OpenLocale() failed to actually open the locale, having allocated
  memory for it first, it would have freed that memory twice. This appears
  to be a bug which goes back to at least 1992. Fixed.


locale 47.10 (19.9.2020)

- If there are at least 128 entries in a catalog file, locale library
  will now try to build a lookup table for the catalog strings, sorted
  by string ID at OpenCatalog() time. If enough memory is available,
  that is. GetCatalogStr() will then make use of this table to perform
  a binary search for the catalog strings. This should make lookups for
  large catalogs noticeably faster, but benchmarking is still needed...

- Added a non-recursive local qsort() implementation in order to avoid
  pulling in the entire qsort() and stack management code for them
  SAS/C runtime library.


locale 47.11 (20.9.2020)

- Fix for free after free -- a classic copy & paste error. Thanks go
  to Thomas Klein for finding and reporting it.


locale 47.12 (20.9.2020)

- Changed the optimization options for the build so as not to
  inline local functions. This is to limit the amount of stack
  space used.

- Moved the locale initialization out of the LoadLocale() function
  in order to simplify the code.


locale 47.13 (20.9.2020)

- If loading a catalog from disk fails because there is not enough
  memory available, we now check if the catalog has already been
  loaded (by a different client) before we give up. It might just
  work...

- Moved three separate instances of the code which tries to reuse an
  existing, cached catalog into a single separate function which can
  be conveniently reused.

- Freeing unused catalogs did not free the catalog entry search
  table. Fixed.


locale 47.14 (21.9.2020)

- Updated the code documentation.

- Changed the names of certain function parameters as well as local
  variables so as to better convey their respective purposes.

- CloseLocale() no longer decrements the usage counter of a locale
  if its counter is already 0.


locale 47.15 (22.12.2020)

- Added experimental replacement code for the FormatString() function
  which is hopefully more robust and also faster than the original
  implementation. This replacement is not enabled by default and
  requires recompilation of the "format.c" file with the preprocessor
  symbol "USE_NEW_FORMATSTRING" defined.


locale 47.16 (23.12.2020)

- Now stores the actual sizes of the parameters instead of 1 (short)
  and 2 (long) and later multiplying that by 2. This actually saved
  a few bytes of code.

- Pointer type casting tricks made the different conversions for
  signed and unsigned 16 bit integer/character parameters unnecessary.

- Updated the AutoDocs to mention that in all version including 47.14
  the string produced by the %s conversion specifier was limited to a
  maximum of 65535 characters. This limitation has been removed.


locale 47.17 (23.12.2020)

- If possible, NUL-terminated strings are now copied to the output
  until the '\0' character is found rather than first determining
  the length of the string. This way, each single character is accessed
  exactly once (not twice).


locale 47.18 (23.12.2020)

- Updated the FormatString() documentation to include a warning to the
  effect that when specifying argument positions (as in "%5$s") only up
  to 128 arguments may be used.

- Rewrote format_uses_position_spec() not to use the index variable
  i on the string, but to modify the string pointer instead. That made
  the code noticeably shorter, perhaps faster, too.

- Rewrote the initial scanning loop in format_string_with_position_arguments()
  not to use the index variable i on the format string, but to modify the
  string pointer instead. Again, that made the code shorter, perhaps faster.

- Rewrote format_string_without_position_arguments() to modify the format
  string pointer instead of using the index variable in on it.


locale 47.19 (23.12.2020)

- Removed unused code from the format_string_without_position_arguments()
  function. It will never get called with a "%<number>$s" conversion
  specification anyway, so we don't have to look for the "$" character.

- Reduced the stack size requirements for FormatString() by cutting down
  the number conversion buffer to 100 characters (from 200). As Patrik
  Axelsson proved, the worst case never touches the 100 character limit
  and the recently updated code always checks if it has reached the
  limits of the number conversion buffer.

- Changed format_string_with_position_arguments() so that it dynamically
  fills the parameter offset array with zeroes as new parameters are
  tracked. This should be more efficient than always filling the entire
  buffer with zeroes when the function is invoked. Patrik showed that this
  approach will significantly reduce the cost of calling this function.

- Updated and corrected the documentation (again).


locale 47.20 (25.12.2020)

- Modified the code which inserts the number grouping characters so that
  it references the array in locale->el_NumFormat through a local pointer,
  which generates shorter and maybe faster code.

  The next step was to rework the code which accesses this array and which
  ought to make sure that it does not overrun its boundaries. The boundary
  check is now much simpler in operation and it watches both for over- and
  underrun.


locale 47.21 (2.1.2021)

- Permanently enabled the replacement code for FormatString(), which is
  more performant and has predictable result if internal limits are
  breached (for example number of reordered arguments).

  To get an idea of the performance difference compared to the old code,
  lets show some benchmark numbers for exec.library/RawDoFmt(). The reason
  for measuring RawDoFmt() is because it is the main use for FormatString().
  When locale.library is loaded, it patches exec.library to use
  FormatString() to handle RawDoFmt().

  Some A3000 030@25Mhz benchmarks for RawDoFmt() with plain strings, where:
  - Iter:         Number of iterations of the test
  - Len:          Length of the string used as input to the function
  - Before:       Runtime in seconds for old code.
  - After:        Runtime in seconds for the new code.
  - Ratio:        Ratio between the runtime of the old and the new
                  replacement code.

    Iter    Len  Before  After  Ratio
     160    625   1.017  1.049  0.969
     800    125   1.186  1.065  1.113
    4000     25   2.049  1.213  1.689
   20000      5   6.183  1.899  3.255
  100000      1  27.027  5.341  5.060

  As can be seen, the large initial cost has been reduced significantly.


locale 47.22 (9.1.2021)

- Mixing conversion specifications with and without a specific argument
  position with FormatString() was always unreliable and could lead to
  undefined behaviour by "consuming" more parameter data than was provided.
  The documentation now reflects this and the implementation tries to bring
  some consistency to the order in which the parameter data is used.


locale 47.23 (15.2.2021)

- Turns out that some software, such as TurboCalc 3.5, depends upon the
  GetCatalogStr() function to return with a specific register configuration.
  If this configuration does not match how locale.library V38-V40 would
  set it up, undefined behaviour would follow. locale.library now makes
  sure that registers D1, A0 and A1 match the locale.library V38-V40
  behaviour. Note that D1, A0 and A1 are scratch registers and no code
  should depend upon their contents unless the documentation states
  that this configuration is to be expected.

- Updated the BUGS section of the GetCatalogStr() function, concerning
  the accidental A0 and A1 register configuration.

- Removed redundant tests from the LoadCatalog() function which saved
  some 40 bytes.


locale 47.24 (16.2.2021)

- Small corrections to the documentation in "getcatalogstrwrapper.asm".

- Smaller changes to the catalog processing which hopefully result in
  better code readability.

- Updated the catalog processing code to verify if the catalog data could
  be read and if the catalog contains a valid language chunk. If none of these
  are present (e.g. the language is missing, is not NUL-terminated or an empty
  string) then the catalog will be rejected as faulty. Furthermore, if there is
  no usable version chunk in the catalog and a specific version of that catalog
  was requested, then the catalog will be rejected as faulty, too (you can't
  check the version number if the data to check isn't there). Finally, the
  catalog string information itself is subjected to tests which insure that
  no zero length strings are permitted, strings can never be larger than
  what the catalog string buffer holds and numeric overflows before rounding
  up the total string size are detected early on.

  The point of this exercise is to curb undefined behaviour. Previously, the
  catalog processing code assumed that all the chunks were present and trusted
  that their respective contents were sane.


locale 47.25 (7.3.2021)

- The debug version of locale.library (which will be the default build until
  we ship the release version or decide differently) can be instructed to
  produce debug output via kprintf() if a catalog file cannot be loaded. This
  is done by setting either a local or global environment variable called
  "locale.library.debug", like so: setenv locale.library.debug 1
  If the variable is not present or contains a value < 1 then no debug output
  will be produced. Thanks go to Robert Miranda who suggested this feature,
  which should help in identifying catalog files which do not pass consistency
  checks or which cannot be loaded for other reasons (e.g. out of memory).


locale 47.26 (11.4.2021)

- Now handles catalogs with empty STRS chunks robustly, as well as catalog
  files which lack a STRS chunk.

- It is no longer considered an error if a catalog file has no valid
  FVER chunk or even no FVER chunk in the first place. However, if the
  client asked for a specific version of the catalog file missing or
  invalid FVER chunks will be treated as if the catalog version did
  not match the requested version.
