Springe zum Hauptinhalt

Using languagetool with a local server

Kommentare

Source

Tags:

language-tool-logo.png

LanguageTool Download

Follow Link "Open-source development" - and then "How to run your own LanguageTool server". In the section "Getting the server" there is a download link.

Installing fastText

In order to efficiently discriminate between 176 different languages, we need to install fasttext and its language identification model: 

dzu@krikkit:~$ sudo apt install fasttext
Installing:                     
  fasttext

Summary:
  Upgrading: 0, Installing: 1, Removing: 0, Not Upgrading: 0
  Download size: 77.4 kB
  Space needed: 179 kB / 61.6 GB available

Get:1 http://deb.debian.org/debian trixie/main amd64 fasttext amd64 0.9.2+ds-7+b1 [77.4 kB]
Fetched 77.4 kB in 0s (643 kB/s)
Selecting previously unselected package fasttext.
(Reading database ... 668886 files and directories currently installed.)
Preparing to unpack .../fasttext_0.9.2+ds-7+b1_amd64.deb ...
Unpacking fasttext (0.9.2+ds-7+b1) ...
Setting up fasttext (0.9.2+ds-7+b1) ...
Processing triggers for man-db (2.12.1-2) ...
dzu@krikkit:~$ cd /tmp 
dzu@krikkit:/tmp$ wget https://dl.fbaipublicfiles.com/fasttext/supervised-models/lid.176.bin
--2024-08-05 23:56:40--  https://dl.fbaipublicfiles.com/fasttext/supervised-models/lid.176.bin
Resolving dl.fbaipublicfiles.com (dl.fbaipublicfiles.com)... 2600:9000:223e:ea00:13:6e38:acc0:93a1, 2600:9000:223e:800:13:6e38:acc0:93a1, 2600:9000:223e:2200:13:6e38:acc0:93a1, ...
Connecting to dl.fbaipublicfiles.com (dl.fbaipublicfiles.com)|2600:9000:223e:ea00:13:6e38:acc0:93a1|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 131266198 (125M) [application/octet-stream]
Saving to: 'lid.176.bin'

lid.176.bin                 100%[=========================================>] 125.18M  95.2MB/s    in 1.3s    

2024-08-05 23:56:42 (95.2 MB/s) - 'lid.176.bin' saved [131266198/131266198]

dzu@krikkit:/tmp$ sudo mv lid.176.bin /usr/local/share/LanguageTool-6.4/
dzu@krikkit:/tmp$

Finally we need to save the configuration to a a config file: 

dzu@krikkit:/tmp$ cd /usr/local/share/LanguageTool-6.4/
dzu@krikkit:/usr/local/share/LanguageTool-6.4$ cat <<EOF | sudo tee server.properties > /dev/null
fasttextModel=//usr/local/share/LanguageTool-6.4/lid.176.bin
fasttextBinary=/usr/bin/fasttext
EOF
dzu@krikkit:/usr/local/share/LanguageTool-6.4$ cat server.properties 
fasttextModel=//usr/local/share/LanguageTool-6.4/lid.176.bin
fasttextBinary=/usr/bin/fasttext
dzu@krikkit:/usr/local/share/LanguageTool-6.4$

LanguageTool HTTP Server

https://dev.languagetool.org/http-server 

dzu@krikkit:~$ java -cp /usr/local/share/LanguageTool-6.4/languagetool-server.jar \
  org.languagetool.server.HTTPServer \
  --config /usr/local/share/LanguageTool-6.4/server.properties \
  --port 8081 --allow-origin
2024-08-06 00:04:19.490 GMT+02:00 INFO  org.languagetool.server.DatabaseAccessOpenSource Not setting up database access, dbDriver is not configured
2024-08-05 22:04:19 +0000 Setting up thread pool with 10 threads
2024-08-06 00:04:20.753 GMT+02:00 INFO  org.languagetool.language.identifier.DefaultLanguageIdentifier Started fastText process for language identification: Binary /usr/bin/fasttext with model @ /usr/local/share/LanguageTool-6.4/lid.176.bin
2024-08-06 00:04:20.759 GMT+02:00 INFO  org.languagetool.server.TextChecker A/B-Test enabled: []
2024-08-05 22:04:20 +0000 Starting LanguageTool 6.4 (build date: 2024-03-28 14:05:28 +0100, 0e9362b) server on http://localhost:8081...
2024-08-05 22:04:20 +0000 Server started

Language tool cmdline check

dzu@krikkit:~$ curl -d "language=en-US" -d "text=a simple test" http://localhost:8081/v2/check | jq .
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1216  100  1183  100    33    292      8  0:00:04  0:00:04 --:--:--   300
{
  "software": {
    "name": "LanguageTool",
    "version": "6.4",
    "buildDate": "2024-03-28 14:05:28 +0100",
    "apiVersion": 1,
    "premium": false,
    "premiumHint": "You might be missing errors only the Premium version can find. Contact us at support<at>languagetoolplus.com.",
    "status": ""
  },
  "warnings": {
    "incompleteResults": false
  },
  "language": {
    "name": "English (US)",
    "code": "en-US",
    "detectedLanguage": {
      "name": "English (US)",
      "code": "en-US",
      "confidence": 0.429,
      "source": "fasttext+commonwords"
    }
  },
  "matches": [
    {
      "message": "This sentence does not start with an uppercase letter.",
      "shortMessage": "",
      "replacements": [
        {
          "value": "A"
        }
      ],
      "offset": 0,
      "length": 1,
      "context": {
        "text": "a simple test",
        "offset": 0,
        "length": 1
      },
      "sentence": "a simple test",
      "type": {
        "typeName": "Other"
      },
      "rule": {
        "id": "UPPERCASE_SENTENCE_START",
        "description": "Checks that a sentence starts with an uppercase letter",
        "issueType": "typographical",
        "urls": [
          {
            "value": "https://languagetool.org/insights/post/spelling-capital-letters/"
          }
        ],
        "category": {
          "id": "CASING",
          "name": "Capitalization"
        }
      },
      "ignoreForIncompleteSentence": true,
      "contextForSureMatch": -1
    }
  ],
  "sentenceRanges": [
    [
      0,
      13
    ]
  ],
  "extendedSentenceRanges": [
    {
      "from": 0,
      "to": 13,
      "detectedLanguages": [
        {
          "language": "en",
          "rate": 1.0
        }
      ]
    }
  ]
}
dzu@krikkit:~$

Firefox Addon

language-tool-firefox.png

LanguageTool as a Language Server

https://github.com/valentjn/ltex-ls

Emacs and eglot

https://github.com/emacs-languagetool/eglot-ltex 

[client-request] (id:9) Wed Oct 19 12:40:44 2022:
(:jsonrpc "2.0" :id 9 :method "workspace/executeCommand" :params
          (:command "_ltex.addToDictionary" :arguments
                    [(:uri "file:///home/dzu/blog/posts/linux/switching-to-secureboot.org" :words
                           (:en-US
                            ["exfiltrators"]))]))
[server-reply] (id:9) Wed Oct 19 12:40:44 2022:
(:jsonrpc "2.0" :id 9 :result
          (:success :json-false :errorMessage "Unknown command '_ltex.addToDictionary', ignoring"))

systemd socket activation

~/.config/systemd/user/languagetool-server.socket

[Unit]
Description=LanguageTool Singleton Socket

[Socket]
ListenStream=8081
Service=languagetool-server-proxy.service

[Install]
WantedBy=sockets.target

~/.config/systemd/user/languagetool-server-proxy.service

[Unit]
Description=LanguageTool Server Proxy
Requires=languagetool-server.service
After=languagetool-server.service
JoinsNamespaceOf=languagetool-server.service

[Service]
ExecStart=/lib/systemd/systemd-socket-proxyd 127.0.0.1:8082
PrivateTmp=yes

~/.config/systemd/user/languagetool-server.service

[Unit]
Description=LanguageTool HTTP Server

[Service]
ExecStart=java -cp /usr/local/share/LanguageTool-6.4/languagetool-server.jar org.languagetool.server.HTTPServer --port 8082 --allow-origin
# Wait for the HTTP server to be available
ExecStartPost=/usr/bin/sleep 2.5
PrivateTmp=yes

Installing stuff

systemctl --user enable languagetool-server.socket
systemctl --user start languagetool-server.socket

Kommentare

Comments powered by Disqus