Articut

Introduction to the Articut Service

Articut is a Word-Segmentation/POS-tagging system based on "Syntactic Rules."

Since Articut is based on syntax structure, it does not require BigData for training. Accuracy improvement takes just few minutes and can be deployed offline. Articut does not include a dictionary. That enables the ability to inference new words.

In addition to Word-Segmentation, Articut is also POS-tagging system. It calculates Segmentation + Part-of-Speech + Named Entity Recognition (NER)
This helps computers to extract words as their minimal meaning units in the sentence.

How to Launch Articut

Please log in to your Droidtown API Member Profile, go to the Articut tab, and under the Docker Packaging section, check the [Launcher] option and select the versions of [Articut] and [LokiTool] you need. Then, click the Download button. (The file name will be ArticutDocker.zip)

After unzipping, place all files in the same directory. Please refer to the following directory structure:

Step 1

[Docker]

$ sudo gpasswd -a $USER docker

Please make sure that the [enabled account] on the server has been added to the Docker group, so you can use Docker commands without needing sudo.
If it hasn’t been set, please enter the shell command shown on the right:

[nerdctl]

$ containerd-rootless-setuptool.sh install

Please ensure that the server's [enabled account] has rootless nerdctl installed. Reference Documentation

Step 2

# sudo apt install net-tools    # Ubuntu
# sudo yum install net-tools    # RHEL
$ ifconfig

Please confirm the following information (this is the host information received by Droidtown):

[MAC ADDRESS]

Name of the physical network interface (must not be lo or Docker's virtual network interface)
MAC_ADDRESS etherxx:xx:xx:xx:xx:xx

[Location of the Host's SSH KEY]

USERNAME
SSH_PRIVATE_KEY /home/{USERNAME}/.ssh/{SSH_PRIVATE_KEY}

Step 3

Launch ArticutStarter

$ ./ArticutEnStarter

Start ArticutStarter. The first launch may take a bit longer (about 2–3 minutes) because it needs to build the Articut Docker image. From the second launch onward, it will start Articut directly without rebuilding the image.

CPU core = 4， Worker = 2 (maximum can be set to CPU * 2 = 8)

$ ./ArticutEnStarter -w 2

If you want to use multiple Worker, you can add the -w {Num} parameter when launching. {Num} must not exceed the number of CPU cores × 2.

Launch Successful

Launch Failed

Step 4

Install ArticutAPI

python3 -m pip install ArticutAPI

Download ArticutAPI

git clone git@github.com:Droidtown/ArticutAPI.git

Test whether the Articut Docker on the server is working properly.

First, install ArticutAPI or download it from https://github.com/Droidtown/ArticutAPI
Test Articut Docker

from ArticutAPI.MP_ArticutAPI import MP_Articut

PORT = 8964
URL = "127.0.0.1"
articut = MP_Articut(url=URL, port=PORT)
result = articut.parse("Initialization complete. Articut Docker Pro system is operating normally.")

print(result)

Please change the URL variable to your Domain Name or IP address, Port: 8964.

Update ArticutDocker

$ ./ArticutEnStarter -l

Please log in to your Droidtown API Member Profile, go to the Articut tab, and under the Docker Packaging section, select the version you want to use and download ArticutDocker.zip. After unzipping, overwrite all files in the original ArticutDocker folder with the new ones, and execute the shell command on the right:

Set Up Articut Service

which nerdctl
# /home/droidtown/.local/bin/nerdctl

echo $XDG_RUNTIME_DIR
# /run/user/1000

Modify ArticutEnServiceChecker.py

If you are using nerdctl to start Articut, modify the two variables (pathLIST and XDG_RUNTIME_DIR) in ArticutEnServiceChecker.py. For the Docker version, you can skip this setup. - Get the environment variable $XDG_RUNTIME_DIR location, and modify the os.environ ["XDG_RUNTIME_DIR"] variable.
- If nerdctl is not installed under /usr/local/bin, add the nerdctl path to pathLIST.

crontab -e

# This example checks whether the Articut Service is functioning properly every minute.
# ARTICUT_WORKING_PATH is the location where Articut operates.
*/1 * * * * python3 /ARTICUT_WORKING_PATH/ArticutEnServiceChecker.py >> /ARTICUT_WORKING_PATH/logs/ArticutEnServiceChecker.log 2>&1

Set Check Time

Since Articut Docker Pro runs under Containerd, you can use either crontab or systemd to set the check time. By default, it checks every minute.

crontab：
Use crontab to set the time for regularly checking if the Articut Service is running properly. You can adjust the check time based on your needs. If the Articut Service does not exist, it will automatically restart.

articut_en.service

User=USERNAME   # The USERNAME on your host that starts Articut
Group=GROUP     # The GROUP on your host that starts Articut

# WorkingDirectory and ExecStart must use absolute paths
WorkingDirectory=ARTICUT_WORKING_PATH    # Articut's working directory
ExecStart=/bin/sh -c "python3 ARTICUT_WORKING_PATH/ArticutEnServiceChecker.py"

sudo cp ./articut_en.service /etc/systemd/system/articut_en.service
sudo cp ./articut_en.timer /etc/systemd/system/articut_en.timer
sudo systemctl daemon-reload
sudo systemctl enable articut_en.timer    # Enable to start on boot
sudo systemctl start articut_en.timer     # Start articut_en.timer

sudo systemctl list-timers             # Check if articut_en.timer is in the schedule

systemd：
Please open articut_en.service and modify the ARTICUT_WORKING_PATH to set Articut's working directory. If you want to adjust the check time, modify articut_en.timer. For the time format, please refer to systemd timer。
Place articut_en.service and articut_en.timer in the /etc/systemd/system/ directory, and execute the shell command on the right:

Check if articut_en.timer has been added to the schedule:

Articut Advanced Settings

The advanced configuration file settings.conf is located in the ArticutDocker directory. You can adjust the number of workers (recommended based on the number of CPUs), timeout, port location (default is 8964), etc., according to your machine's specifications.

The configuration is shown in the image below:

For parameter explanations, please refer to: Gunicorn Doc

Using the Articut API

lv2 Sample Code:

from requests import post

url = "http://ARTICUT_URL/Articut_EN/API/"
payload = {
    "input_str": "Colorless green ideas sleep furiously."
}

response = post(url, json=payload)

var url = "http://ARTICUT_URL/Articut_EN/API/";

var xhr = new XMLHttpRequest();
xhr.open("POST", url);

xhr.setRequestHeader("Accept", "application/json");
xhr.setRequestHeader("Content-Type", "application/json");

xhr.onreadystatechange = function () {
    if (xhr.readyState === 4) {
        console.log(xhr.status);
        console.log(xhr.responseText);
    }
};

var data = `{
    "input_str":"Colorless green ideas sleep furiously."
}`;

xhr.send(data);

Returned Result (JSON Format):

{
    "exec_time": 0.01952219009399414,
    "result_pos": ["<MODIFIER>Colorless</MODIFIER> <MODIFIER>green</MODIFIER> <ENTITY_nouny>ideas</ENTITY_nouny> <ACTION_verb>sleep</ACTION_verb> <MODIFIER>furiously</MODIFIER>", "."],
    "result_segmentation": ["Colorless/green/ideas/sleep/furiously/."],
    "result_obj": [[{"text": "Colorless", "pos": "MODIFIER"},
                    {"text": "green", "pos": "MODIFIER"},
                    {"text": "ideas", "pos": "ENTITY_nouny"},
                    {"text": "sleep", "pos": "ACTION_verb"},
                    {"text": "furiously", "pos": "MODIFIER"}],
                   [{"text": ".","pos": "PUNCTUATION"}]],
    "level": "lv2",
    "version": "v220",
    "status": true,
    "msg": "Success!",
    "info": "ArticutEN Docker Pro only for Droidtown. Powered by Droidtown Linguist Tech."
}

HTTP Request

POST http://ARTICUT_URL/Articut_EN/API/

Arguments

Argument	Type	Default	Description
input_str	str	""	The text to be sent to Articut for word segmentation and part-of-speech tagging.
level	str	"lv2"	Currently, Articut only provide lv2 for English texts.
user_defined_dict_file	dict	{}	User-defined dictionary in dictionary format. (e.g. UserDefinedDICT = {"key": ["value1", "value2",...],...})。
emoji	bool	True	Articut can detects emoji symbols and marks them as `<ENTITY_oov>`. `If the text does not contain emojis, disabling this can improve processing speed.`

Result Returned

Returned Message	Type	Description
status	bool	Returns True if the word segmentation process is successfully executed and the results are received; returns False if it fails.
msg	str	Possible messages:
		- Success!: The word segmentation task was completed successfully.
		- Authtication failed.: There seems to be an issue with your authentication. Please contact Droidtown for assistance.
		- Internal server error. (Your word count balance is not consumed, don't worry. System will reboot in 5min, please try again later.): Hmm... it seems there was an issue. The system will automatically restart within 5 minutes, so please try again later. If the issue persists, please contact Droidtown to resolve it.
		- Invalid content_type.: The upload format must be in JSON format (application/json).
		- Invalid arguments.: There was an error with the uploaded arguments. Please check if the rguments conform to the correct naming rules.
		- UserDefinedDICT Parsing ERROR. (Please check your the format and encoding.): The user-defined dictionary could not be loaded. Please check if the format (Dict) or encoding (UTF-8) is correct.
		- Maximum UserDefinedDICT file size exceeded! (UserDefinedDICT file shall be samller than 10MB.): The user-defined dictionary file size exceeds 10MB.
result_pos	list	A list containing the word segmentation results for each sentence, with Part-Of-Speech (POS) tags added before and after each phrase.
result_obj	list	A list containing the word segmentation object results for each sentence, including both the phrases and their POS tags.
result_segmentation	str	The complete input sentence has been segmented, with word boundaries marked by slashes ( / ). The result is returned as a string.
exec_time	float	The server time consumed for this word segmentation task.
version	str	The algorithm version used for this word segmentation task.
level	str	The knowledge level used for this word segmentation task.

Using Articut Bulk API

lv2 Sample Code:

from requests import post

url = "http://ARTICUT_URL/Articut_EN/BulkAPI/"
payload = {
    "input_list": ["Colorless green ideas sleep furiously.", "I saw the man with the telescope."]
}

response = post(url, json=payload)

var url = "http://ARTICUT_URL/Articut_EN/BulkAPI/";

var xhr = new XMLHttpRequest();
xhr.open("POST", url);

xhr.setRequestHeader("Accept", "application/json");
xhr.setRequestHeader("Content-Type", "application/json");

xhr.onreadystatechange = function () {
    if (xhr.readyState === 4) {
        console.log(xhr.status);
        console.log(xhr.responseText);
    }
};

var data = `{
    "input_list": ["Colorless green ideas sleep furiously.", "I saw the man with the telescope."]
}`;

xhr.send(data);

Returned Result (JSON Format):

{
    "result_list": [
        {"exec_time": 0.01952219009399414,
         "result_pos": ["<MODIFIER>Colorless</MODIFIER> <MODIFIER>green</MODIFIER> <ENTITY_nouny>ideas</ENTITY_nouny> <ACTION_verb>sleep</ACTION_verb> <MODIFIER>furiously</MODIFIER>", "."],
         "result_segmentation": ["Colorless/green/ideas/sleep/furiously/."],
         "result_obj": [[{"text": "Colorless", "pos": "MODIFIER"},
                         {"text": "green", "pos": "MODIFIER"},
                         {"text": "ideas", "pos": "ENTITY_nouny"},
                         {"text": "sleep", "pos": "ACTION_verb"},
                         {"text": "furiously", "pos": "MODIFIER"}],
                         [{"text": ".","pos": "PUNCTUATION"}]]},
        {"exec_time": 0.017029762268066406,
         "result_pos": ["<ENTITY_pronoun>I</ENTITY_pronoun> <ACTION_verb>saw</ACTION_verb> <FUNC_determiner>the</FUNC_determiner> <ENTITY_nouny>man</ENTITY_nouny> <FUNC_inner>with</FUNC_inner> <FUNC_determiner>the</FUNC_determiner> <ENTITY_nouny>telescope</ENTITY_nouny>", "."],
         "result_segmentation": ["I/saw/the/man/with/the/telescope/."],
         "result_obj": [[{"text": "I", "pos": "ENTITY_pronoun"},
                         {"text": "saw", "pos": "ACTION_verb"},
                         {"text": "the", "pos": "FUNC_determiner"},
                         {"text": "man", "pos": "ENTITY_nouny"},
                         {"text": "with", "pos": "FUNC_inner"},
                         {"text": "the", "pos": "FUNC_determiner"},
                         {"text": "telescope", "pos": "ENTITY_nouny"}],
                        [{"text": ".", "pos": "PUNCTUATION"}]]}],
    "level": "lv2",
    "version": "v220",
    "status": true,
    "msg": "Success!",
    "info": "ArticutEN Docker Pro only for Droidtown. Powered by Droidtown Linguist Tech."
}

HTTP Request

POST http://ARTICUT_URL/Articut_EN/BulkAPI/

Arguments

Argument	Type	Default	Description
input_str	str	""	The text to be sent to Articut for word segmentation and part-of-speech tagging.
level	str	"lv2"	Currently, Articut only provide lv2 for English texts.
user_defined_dict_file	dict	{}	User-defined dictionary in dictionary format. (e.g. UserDefinedDICT = {"key": ["value1", "value2",...],...})。
emoji	bool	True	Articut can detects emoji symbols and marks them as `<ENTITY_oov>`. `If the text does not contain emojis, disabling this can improve processing speed.`

Result Returned

Returned Message	Type	Description
status	bool	Returns True if the word segmentation process is successfully executed and the results are received; returns False if it fails.
msg	str	Possible messages:
		- Success!: The word segmentation task was completed successfully.
		- Authtication failed.: There seems to be an issue with your authentication. Please contact Droidtown for assistance.
		- Internal server error. (Your word count balance is not consumed, don't worry. System will reboot in 5min, please try again later.): Hmm... it seems there was an issue. The system will automatically restart within 5 minutes, so please try again later. If the issue persists, please contact Droidtown to resolve it.
		- Invalid content_type.: The upload format must be in JSON format (application/json).
		- Invalid arguments.: There was an error with the uploaded arguments. Please check if the rguments conform to the correct naming rules.
		- UserDefinedDICT Parsing ERROR. (Please check your the format and encoding.): The user-defined dictionary could not be loaded. Please check if the format (Dict) or encoding (UTF-8) is correct.
		- Maximum UserDefinedDICT file size exceeded! (UserDefinedDICT file shall be samller than 10MB.): The user-defined dictionary file size exceeds 10MB.
result_pos	list	A list containing the word segmentation results for each sentence, with Part-Of-Speech (POS) tags added before and after each phrase.
result_obj	list	A list containing the word segmentation object results for each sentence, including both the phrases and their POS tags.
result_segmentation	str	The complete input sentence has been segmented, with word boundaries marked by slashes ( / ). The result is returned as a string.
exec_time	float	The server time consumed for this word segmentation task.
version	str	The algorithm version used for this word segmentation task.
level	str	The knowledge level used for this word segmentation task.

Get the current version

Sample code:

from requests import get

url = "http://ARTICUT_URL/Articut_EN/Version/"

response = get(url)

var url = "http://ARTICUT_URL/Articut_EN/Version/";

var xhr = new XMLHttpRequest();
xhr.open("GET", url);

xhr.setRequestHeader("Accept", "application/json");

xhr.onreadystatechange = function () {
   if (xhr.readyState === 4) {
      console.log(xhr.status);
      console.log(xhr.responseText);
   }};

xhr.send();

Returned Result (JSON Format):

{
    "status": true,
    "version": "v220",
    "msg": "Success!",
    "info": "ArticutEN Docker Pro only for Droidtown. Powered by Droidtown Linguist Tech."
}

HTTP Request

GET http://ARTICUT_URL/Articut_EN/Version/

Result Returned

Returned message	Type	Description
status	bool	Returns True if execution is successful and the result is retrieved; returns False if it fails.
msg	str	May contain one of the following messages:
		- Success!: Successfully retrieved the version number.
		- Authtication failed.: There seems to be an issue with your authentication. Please contact Droidtown for assistance.
		- Internal server error. (Your word count balance is not consumed, don't worry. System will reboot in 5min, please try again later.): Hmm... it seems there was an issue. The system will automatically restart within 5 minutes, so please try again later. If the issue persists, please contact Droidtown to resolve it.
version	str	The currently used version of Articut.

Use UserDefinedDICT

Sample code:

from requests import post

url = "http://ARTICUT_URL/Articut_EN/API/"
payload = {
    "input_str": "I am planning the Human Instrumentality Project",
    "user_defined_dict_file": {"Human Instrumentality Project": ["Instrumentality Project"]}
}

response = post(url, json=payload)

var url = "http://ARTICUT_URL/Articut_EN/API/";

var xhr = new XMLHttpRequest();
xhr.open("POST", url);

xhr.setRequestHeader("Accept", "application/json");
xhr.setRequestHeader("Content-Type", "application/json");

xhr.onreadystatechange = function () {
    if (xhr.readyState === 4) {
        console.log(xhr.status);
        console.log(xhr.responseText);
    }
};

var data = `{
    "input_str": "I am planning the Human Instrumentality Project",
    "user_defined_dict_file": {"Human Instrumentality Project": ["Instrumentality Project"]}
}`;

xhr.send(data);

Returned Result (JSON Format):

{
    "exec_time": 0.9545776844024658,
    "result_pos": ["<ENTITY_pronoun>I</ENTITY_pronoun> <AUX>am</AUX> <ACTION_verb>planning</ACTION_verb> <FUNC_determiner>the</FUNC_determiner> <UserDefined>Human Instrumentality Project</UserDefined>"],
    "result_segmentation": ["I/am/planning/the/Human Instrumentality Project"],
    "result_obj": [[{"text": "I", "pos": "ENTITY_pronoun"},
            {"text": "am", "pos": "AUX"},
                    {"text": "planning", "pos": "ACTION_verb"},
                    {"text": "the", "pos": "FUNC_determiner"},
                    {"text": "Human Instrumentality Project", "pos": "UserDefined"}]],
    "level": "lv2",
    "version": "v220",
    "status": true,
    "msg": "Success!",
    "info": "ArticutEN Docker Pro only for Droidtown. Powered by Droidtown Linguist Tech."
}

Because Articut only processes "language knowledge" and does not handle "encyclopedic knowledge". We provide a feature for "user-defined" vocabulary lists, which are marked as <UserDefined>. Use the Dictionary format and write it yourself.

HTTP Request

POST http://ARTICUT_URL/Articut_EN/API/

Arguments

Argument	Type	Default	Description
input_str	str	""	The text to be sent to Articut for segmentation and part-of-speech tagging.
user_defined_dict_file	dict	{}	User-defined dictionary, must be in dictionary format. (e.g. UserDefinedDICT = {"key": ["value1", "value2",...],...})。

Result Returned

Returned message	Type	Description
status	bool	Its value will be True or False.
msg	str	Can be one of the following messages:
		- Success!: Segmentation was completed successfully.
		- UserDefinedDICT Parsing ERROR. (Please check your the format and encoding.): The user-defined dictionary could not be loaded, please check if the format (Dict) or encoding (UTF-8) is correct.
		- Maximum UserDefinedDICT file size exceeded! (UserDefinedDICT file shall be samller than 10MB.): The user-defined dictionary file size exceeds 10MB.

Part-of-speech (POS) Tagging

Do(MODAL) not(FUNC_negation) give up(VerbP). We(ENTITY_pronoun) look forward to(VerbP) seeing(ACTION_verb) strong(MODIFIER) AI(ENTITY_nouny) soon(MODIFIER).

Without the idea of POS, words are just a string of symbols connected by space. It takes the knowledge of POS to compile the meanings of sentences. Human understand natural language basing on the POS of the words and its meaning to identify the contribution of them in the sentence.

Tag Categories

Category	Tag
Time	TIME
Entity	ENTITY
Modifier	MODIFIER、QUANTIFIER
Verb	ACTION、ASPECT、MODAL、AUX
Function	FUNC
Clause	CLAUSE
NER	LOCATION、KNOWLEDGE、UserDefined、RANGE

Entity

Entities are usually seen as synonym of noun words. They refer to person, items, things, times, emotions, concepts and directions in both concret and abstrac ways.

When doing NLP with entities, there are few rules you can take advantages of...

Sequential nouny can be taken as a big noun and thus considered as an entity. (e.g., Coffee (ENTITY_nouny) cup (ENTITY_nouny) can be seen as "Coffee cup" (ENTITY_NP)).

Tag	Description
`<ENTITY_num>`	Numeral Expressions
`<ENTITY_classifier>`	Classifier/Measuring methods of something
`<ENTITY_measurement>`	Measuring result (e.g., 10kg)
`<ENTITY_person>`	Entity as a person's name
`<ENTITY_pronoun>`	Entitiy as a pronoun
`<ENTITY_possessive>`	Entity in its possessive case
`<ENTITY_noun>`	Entity as noun
`<ENTITY_nounHead>`	Entity as head of a nominal expression
`<ENTITY_nouny>`	Entity as noun
`<ENTITY_oov>`	Entity as noun
`<ENTITY_DetPhrase>`	Determiner phrase is composed of a determiner and a ordinal number. It can be used as a pronoun referencing to something/someone in the context. e.g., `<ENTITY_DetPhrase>`the first`</ENTITY_DetPhrase>`

Verb

A verb is a word that contribute the main meaning of the sentence. It usually indicates an action, an event or a state of existence.

Tag	Description
`<ACTION_lightVerb>`	Light verbs (e.g., make, let, cause...).
`<ACTION_verb>`	Common verbs.
`<VerbP>`	A phrasal verb consisits of a `Verb` and a `Proposition` or `Functional word` that create a different meaning from either parts of the phrase. e.g.,
	`<VerbP>Look up</VerbP> <FUNC_determiner>the</FUNC_determiner> <ENTITY_nouny>dictionary</ENTITY_nouny>`
	`<VerbP>turn on</VerbP> <FUNC_determiner>the</FUNC_determiner> <ENTITY_nouny>light</ENTITY_nouny>`
`<AUX>`	AUX means auxiliary verbs. Articut tags Be-Verbs as AUX (e.g., is, am, were).
`<MODAL>`	MODAL means modality verbs. It's a semantic marker indicating the ability or permission depending on the contexts (e.g., may, will, should...).

Time

Time/temporal related words/expressions.

Tag	Description
`<TIME_holiday>`	Temporal expressions related to known holidays.
`<TIME_justtime>`	Temporal expressions related to current time or a relatively short period of time.
`<TIME_day>`	Temporal expressions counted by day base.
`<TIME_week>`	Temporal expressions counted by week base.
`<TIME_month>`	Temporal expressions counted by month base.
`<TIME_season>`	Temporal expressions counted by season base.
`<TIME_year>`	Temporal expressions counted by year base.
`<TIME_decade>`	Temporal expressions counted by time that is longer than year base.

Modifiers

Modifiers includes adjectives and adverbs. They are used to modify a sentence, an action, an event in the context or simply a noun.

Articut uses MODIFIER to cover both adjectives and adverbs.

Tag	Description
`<MODIFIER>`	Adjectives and adverbs.
`<QUANTIFIER>`	Quantification markers such "all", "some"...etc.

Function Words

Functions words indicate a word that plays as a syntactical function in a sentence or between sentences.

Tag	Description
`<FUNC_conjunction>`	Conjunction words such as "and"
`<FUNC_determiner>`	Determiners are words placed in front of a noun to make it clear what the noun refers to. Such as "the", "that"...etc.
`<FUNC_inner>`	Function words that indicates some in-sentence semantics. (e.g., "in" may indicate a relation between an action and a place in `I live in this town.`)
`<FUNC_inter>`	Function words that indicates some inter-sentence semantics. (e.g., "however" may indicate an existence of other sentence. (e.g., `I usually don't eat breakfast. However, this toast is just too delicious.`)
`<FUNC_negation>`	Negation words

Clause

Interrogative sentences (wh-questions and yes-no questions) or declarative sentences.

Tag	Description
`<CLAUSE_YesNoQ>`	"Yes-no" question
`<CLAUSE_WhoQ>`	A question about WHO
`<CLAUSE_WhatQ>`	A question about WHAT
`<CLAUSE_WhereQ>`	A question about WHERE
`<CLAUSE_WhenQ>`	A question about WHEN
`<CLAUSE_WhyQ>`	A question about WHY
`<CLAUSE_HowQ>`	A question about the method/degree with HOW
`<CLAUSE_Particle>`	Interjections

Named Entity Recognition (NER)

Named Entiteis means an object in the real world. It could be a place (LOCATION), a person, an organization, a product or proper names of some concret/abstract item. NER helps identify entities that has specific meaning (and thus indicated meaningful features) in a text.

Tag	Description
`<LOCATION>`	Names of a place
`<RANGE_locality>`	Propositions that act as range markers of a place
`<RANGE_period>`	Propositions that act as range markers of a temporal expression
`<UserDefined>`	Strings/Words defiend by users
`<KNOWLEDGE_addUS>`	Address expressions of USA
`<KNOWLEDGE_currency>`	Money with amount `<KNOWLEDGE_currency>100 US dollars</KNOWLEDGE_currency>` or `<KNOWLEDGE_currency>one pound</KNOWLEDGE_currency> <FUNC_conjunction>and</FUNC_conjunction> <KNOWLEDGE_currency>four pence</KNOWLEDGE_currency>`
`<KNOWLEDGE_lawUS>`	Index of legal documents
`<KNOWLEDGE_routeUS>`	Road/Street names
`<KNOWLEDGE_url>`	URL links

LokiServer

Overview of Loki Services

Loki stands for Linguistic Oriented Keyword Interface. It is a next-gen Natural Language Understanding (NLU) engine 。

Loki is driven by syntactic analysis. It converts text into a Python-compatible Regex with condition code blocks.

Comparing to Microsoft LUIS or Google DiaglogFlow, Loki saves tremendous amount of development time and only requires small data.

Loki Solved two problems at the same time:
1. Comparing to other machine-learning block box, the regular expression model generated by Loki is "human-readable/adjustable."
2. No massive amount of data is required!For each utterance pattern, only one occurrance is required to generate a valid regular expression and its conditional block.

Reference Links:
Release Explanation Video for Loki
Advantages of Deploying the Loki Natural Language Understanding Engine
Solving Chinese Math Problems with Loki

Background Information on Loki

Loki directory

├── Your Project1
│   ├── intent_intent1.atm
│   └── intent_intent2.atm
└── Your Project 2
│ ├── intent_intent3.atm
│ └── intnet_intentN.atm
└── Your Project N

After LokiServer is launched along with the ArticutEN Docker (Pro), it will load all <*.atm> models in the Loki directory. You can also deploy models using LokiTool.

Calling Loki API

Sample Code:

from requests import post

url = "http://ARTICUT_URL/Loki_EN/API/"
payload = {
    "input_str": "How much is 100 US dollars in NTD",
    "project": "FinExchange"
} 

response = post(url, json=payload)

var url = "http://ARTICUT_URL/Loki_EN/API/";

var xhr = new XMLHttpRequest();
xhr.open("POST", url);

xhr.setRequestHeader("Accept", "application/json");
xhr.setRequestHeader("Content-Type", "application/json");

xhr.onreadystatechange = function () {
    if (xhr.readyState === 4) {
        console.log(xhr.status);
        console.log(xhr.responseText);
    }
};

var data = `{
    "input_str": "How much is 100 US dollars in NTD",
    "project": "FinExchange"
}`;

xhr.send(data);

Result returned as a JSON object:

{
    "exec_time": 0.3015732765197754,
    "results": [{"intent": "Exchange",
                 "pattern": "<CLAUSE_HowQ>[^<]+</CLAUSE_HowQ>(<QUANTIFIER>[^<]+</QUANTIFIER>)?<AUX>[^<]+</AUX><KNOWLEDGE_currency>100 US dollars</KNOWLEDGE_currency><RANGE_(locality|period)>[^<]+</RANGE_(locality|period)><UserDefined>NTD</UserDefined>",
                 "utterance": "How much is [100 US dollars] in [NTD]",
                 "argument": ["100 US dollars", "NTD"]}],
    "version": "v220",
    "status": true,
    "msg": "Success!",
    "info": "ArticutEN Docker Pro only for Droidtown. Powered by Droidtown Linguist Tech."
}

HTTP Request

POST http://ARTICUT_URL/Loki_EN/API/

Arguments

Argument	Type	Default	Description
input_str	str	""	Texts to be sent to Loki for intent analysis.
project	str	""	Name of the specified project. Loki will analyze all intents within the project.
user_defined_dict_file	dict	{}	User-defined dictionary, must be in dictionary format. `(e.g. UserDefinedDICT = {"key": ["value1", "value2",...],...})。`
filter_list	list	[]	Default as empty list, meaning it will check every intent under the projects. If the list is not empty, Loki will only check the intent specified.

Result Returned

Returned message	Type	Description
status	bool	The value could be True or False。
msg	str	The value could be as listed below:
		- Success!: Request completed successfully.
		- Authentication Failed.: Account (email) and API key doesn't match.
		- Invalid content_type.: The format has to be in Json format (application/json).
		- Invalid arguments.: Argument format error. Please check the types of your arguments.
		- Invalid Project.: loki_key is invalid. Please make sure your loki_key is correct and all projects have been deployed.
		- Internal server error. (Your word count balance is not consumed, don't worry. System will reboot in 5min, please try again later.): Well...it seems there's something wrong with our server. Don't worry, we are fixing it and the server is scheculed to reboo and go online in 5 min. Please try again later. Don't worry, your quota stays intact.
		- Maximum UserDefinedDICT file size exceeded! (UserDefinedDICT file shall be samller than 10MB.): The user-defined dictionary file exceeds the 10MB limit.
		- No Match Intent!: No intent in the project matches the text requested.
		- UserDefinedDICT Parsing ERROR. (Please check your the format and encoding.): Failed to load the user-defined dictionary. Please ensure it is in dict format and encoded in UTF-8.
version	str	Algorithm version used for the request.
results	list	The results of this Loki intent analysis.

Calling Loki Bulk API

Sample Code:

from requests import post

url = "http://ARTICUT_URL/Loki_EN/BulkAPI/"
payload = {
    "input_list": [
        "How much is 100 US dollars in NTD",
        "How much NT can I get for 100 US dollars"
    ],
    "project": "FinExchange"
} 

response = post(url, json=payload)

var url = "http://ARTICUT_URL/Loki_EN/BulkAPI/";

var xhr = new XMLHttpRequest();
xhr.open("POST", url);

xhr.setRequestHeader("Accept", "application/json");
xhr.setRequestHeader("Content-Type", "application/json");

xhr.onreadystatechange = function () {
    if (xhr.readyState === 4) {
        console.log(xhr.status);
        console.log(xhr.responseText);
    }
};

var data = `{
    "input_list": [
        "How much is 100 US dollars in NTD",
        "How much NT can I get for 100 US dollars"
    ],
    "project": "FinExchange"
}`;

xhr.send(data);

Result returned as a JSON object:

{
    "exec_time": 0.3072061538696289,
    "result_list": [
        {"status": true,
         "msg": "Success!",
         "results": [{"intent": "Exchange",
                      "pattern": "<CLAUSE_HowQ>[^<]+</CLAUSE_HowQ>(<QUANTIFIER>[^<]+</QUANTIFIER>)?<AUX>[^<]+</AUX><KNOWLEDGE_currency>100 US dollars</KNOWLEDGE_currency><RANGE_(locality|period)>[^<]+</RANGE_(locality|period)><UserDefined>NTD</UserDefined>",
                      "utterance": "How much is [100 US dollars] in [NTD]",
                      "argument": ["100 US dollars", "NTD"]}]},
        {"status": true,
         "msg": "Success!",
         "results": [{"intent": "Exchange",
                      "pattern": "<CLAUSE_HowQ>[^<]+</CLAUSE_HowQ>(<QUANTIFIER>[^<]+</QUANTIFIER>)?<UserDefined>NT</UserDefined><MODAL>[^<]+</MODAL>(<ENTITY_pronoun>[^<]+</ENTITY_pronoun>)?<(ACTION_verb|VerbP)>get</(ACTION_verb|VerbP)><FUNC_inner>[^<]+</FUNC_inner><KNOWLEDGE_currency>100 US dollars</KNOWLEDGE_currency>",
                      "utterance": "How much [NT] can I get for [100 US dollars]",
                      "argument": ["NT", "100 US dollars"]}]}
    ],
    "version": "v220",
    "status": true,
    "msg": "Success!",
    "info": "ArticutEN Docker Pro only for Droidtown. Powered by Droidtown Linguist Tech."
}

HTTP Request

POST http://ARTICUT_URL/Loki_EN/BulkAPI/

Arguments

Argument	Type	Default	Description
input_list	list	""	Texts to be sent to Loki for intent analysis. `Note: The total number of characters must not exceed 100,000 per request.`
project	str	""	Name of the specified project. Loki will analyze all intents within the project.
user_defined_dict_file	dict	{}	User-defined dictionary, must be in dictionary format. `(e.g. UserDefinedDICT = {"key": ["value1", "value2",...],...})。`
filter_list	list	[]	Default as empty list, meaning it will check every intent under the projects. If the list is not empty, Loki will only check the intent specified.

Result Returned

Result Returned	Type	Description
status	bool	The value could be True or False。
msg	str	The value could be as listed below:
		- Success!: Request completed successfully.
		- No Match Intent!: No intent in the project matches the text requested.
		- Invalid content_type.: The format has to be in Json format (application/json).
		- Invalid arguments.: Argument format error. Please check the types of your arguments.
		- Authentication Failed.: Account (email) and API key doesn't match.
		- Invalid Project.: loki_key is invalid. Please make sure your loki_key is correct and all projects have been deployed.
		- UserDefinedDICT Parsing ERROR. (Please check your the format and encoding.): Failed to load the user-defined dictionary. Please ensure it is in dict format and encoded in UTF-8.
		- Maximum UserDefinedDICT file size exceeded! (UserDefinedDICT file shall be samller than 10MB.): The user-defined dictionary file exceeds the 10MB limit.
		- Internal server error. (Your word count balance is not consumed, don't worry. System will reboot in 5min, please try again later.): Well...it seems there's something wrong with our server. Don't worry, we are fixing it and the server is scheculed to reboo and go online in 5 min. Please try again later. Don't worry, your quota stays intact.
version	str	Algorithm version used for the request.
result_list	list	The results of this Loki intent analysis.

Calling Loki Utterance

Sample Code:

from requests import post

url = "http://ARTICUT_URL/Loki_EN/Utterance/"
payload = {
    "project": "FinExchange"
} 

response = post(url, json=payload)

var url = "http://ARTICUT_URL/Loki_EN/Utterance/";

var xhr = new XMLHttpRequest();
xhr.open("POST", url);

xhr.setRequestHeader("Accept", "application/json");
xhr.setRequestHeader("Content-Type", "application/json");

xhr.onreadystatechange = function () {
    if (xhr.readyState === 4) {
        console.log(xhr.status);
        console.log(xhr.responseText);
    }
};

var data = `{
    "project": "FinExchange"
}`;

xhr.send(data);

Result returned as a JSON object:

{
    "exec_time": 0.1515732765197754,
    "results": {"Exchange": [
        "I want to buy 100 US dollars",
        "How much New Taiwan Dollars do I need for 100 US dollars",
        "How many New Taiwan Dollars can I get for 100 US dollars",
        "How much is 100 US dollars in New Taiwan Dollars",
        "How much NT do I need for 100 US dollars",
        "How much NT can I exchange for 100 US dollars"
    ]},
    "status": true,
    "msg": "Success!",
    "info": "ArticutEN Docker Pro only for Droidtown. Powered by Droidtown Linguist Tech."
}

HTTP Request

POST http://ARTICUT_URL/Loki_EN/Utterance/

Arguments

Arguments	Type	Default	Description
project	str	""	Name of the specified project. Loki will analyze all intents within the project.

Returned message

Returned message	Type	Description
status	bool	The value cloud be True or False。
msg	str	The value could be as listed below:
		- Success!: Request completed successfully.
		- Utterance not found.: No utterance in the intent.
		- Invalid content_type.: The format has to be in Json format (application/json).
		- Authentication Failed.: Account (email) and API key doesn't match.
		- Internal server error. (Your word count balance is not consumed, don't worry. System will reboot in 5min, please try again later.): Well...it seems there's something wrong with our server. Don't worry, we are fixing it and the server is scheculed to reboo and go online in 5 min. Please try again later. Don't worry, your quota stays intact.
version	str	Algorithm version used for the request.
results	list	The results of this Loki intent analysis.

LokiTool

LokiTool is an independent GUI Django website for Loki. It can be deployed with WSGI / ASGI server configurations or copied to a machine that can connect to ArticutEN Docker (Pro) to launch.

How to Launch LokiTool

Install the required packages for LokiTool:

$ pip3 install -r requirements.txt

In the LokiTool directory, make sure your machine has Python 3.8 or above, and install the packages listed in requirements.txt, either by upgrading them to the specified versions or installing those exact versions directly.

Example：

ArticutDocker Server A： http://articut.server_a.com
ArticutDocker Server B： http://articut.server_b.com
ArticutDocker Server C： http://articut.server_c.com

LoadBalancer： http://articut.server.com

Please set the ARTICUT_URL and DEPLOY_URL_LIST as shown below:

ARTICUT_URL = "http://articut.server.com"
DEPLOY_URL_LIST = ["http://articut.server_a.com",
                   "http://articut.server_b.com",
                   "http://articut.server_c.com"]

Set ARTICUT_URL

After ensuring the environment is properly set up, locate the ARTICUT_URL variable in myproject/settings.py and change its value to the address of your ArticutEN Docker (Pro).

If you have purchased multiple instances of ArticutEN Dokcer (Pro)and are using a Load Balancer to connect them (as shown below), be sure to set the DEPLOY_URL_LIST varibel in myproject/settings.py. The system will deploy to all listed Articut Docker (Pro) instances accordingly.

Configure Database Connection

Database connection settings

{
    "//engine": "sqlite / mysql / postgresql",
    "engine": "sqlite",
    "host": "",
    "port": "",
    "database": "",
    "username": "",
    "password": ""
}

Run Database Migration

$ python3 manage.py migrate --fake

If you want to switch to a MySQL or PostgreSQL database, open myproject/database.json, change the engine field to the desired database type (mysql / postgresql), and fill in the database connection details. After making these changes, run a database migration.

Start LokiTool

Start LokiTool

$ python3 manage.py runserver

Next, launch the local LokiTool using manage.py runserver. You can then begin creating NLU projects and writing the contents for each intent.

LokiTool Quickstart Guide

Open LokiTool

In your browser's address bar, enter http://localhost:8000/loki/ to start using LokiTool.

LokiTool

Create a Project

Key-in a project name. The project name should be composed of alphabets [a-z, A-Z] (with digits [0-9] or underline [_]). Click [Create Project]。

Project Name

Use External Synonym API

Synonym API Request (json)

{
    "input_list": ["input1", "input2"]
}

Synonym API Response (json)

{
    "status": true,
    "result": {
        "key": ["val1", "val2"]
    }
}

If you want to connect to a custom synonym API (please follow the format shown on the right)，enter the URL in the External Synonym API field, save it, and then reanalyze all intent sentences.

Synonym API

Create Basic Intent

After entering the [Project], click [Create Basic Intent]。
Key-in an intent name. The project name should be composed of alphabets [a-z, A-Z] (with digits [0-9] or underline [_]). Click [Create Intent]。
If you have a UserDefined Dictionary, you can add it here; if not, proceed to the next step.
After adding the utterance text, click [Analyze] to analyze the syntactic structure/pattern of the utterance. All puncturations in the utterances will be ignored by Loki. Alternatively, you may also enter all the utterance text then click [Analyze All]。
Select (check) the [argument] to represent the intent in the utterance. Usually the arguments indicate words that may vary in this sentence. We will use these selected words for further semantic calculation later.

Loki provides two ways to select intent arguments:

Select [argument] via Graph

Directly select [argument]

PS. If you need to apply the same argument name to multiple items, click < on the right side of the page to open theArgument Selection Tool
Key-in a version number to control the version of your model. Then, click [Deploy Model]。
Enter a testing text in [Testing Tool]to evaluate the model and make sure the arguments in Step 04 are properly arranged.
Repeat Step 04 to Step 06 untill all testing texts can be captured by the model under this Intent。

Create Advanced Intent

After entering the [Project], click [Create Advanced Intent]。
Key-in an intent name. The project name should be composed of alphabets [a-z, A-Z] (with digits [0-9] or underline [_]). Click [Create Intent]。
If you have a UserDefined Dictionary, you can add it here; if not, proceed to the next step.
Enter sentences for the intent and click [Analyze]. The system will generate a default regular expression (Regex). Adjust it as needed, then click[Verify] to ensure the regular expression matches the intent sentences.
Key-in a version number to control the version of your model. Then, click [Deploy Model]。
Enter a sentence and click [Intent Analysis] to test whether the intent can be successfully detected.
Repeat Steps 04 to Step 05 until all training sentences can be correctly matched with their respective intent。

Loading ATM Models

Open LokiTool

Enter http://localhost:8000/loki/ in your browser's address bar to start using LokiTool。

LokiTool

Create a Project

Enter the project name The project name should be composed of alphabets [a-z, A-Z] (with digits [0-9] or underline [_]). Click [Create Project]。

Project Name

Load Models

Load ArticutModel

Under the project section, select "ArticutModel", then click [Browse] > select .ref files (up to 10) > [Load Intent].
Load TXT Plain Text

Under the project section, select "Txt", then click [Browse] > select .txt files (up to 10) > [Load Intent].
Load LUIS Model

Under the project section, select "LUIS", then click [Browse] > select .json files (up to 10) > [Load Intent].
Load DialogFlow Model

Under the project section, select "DialogFlow", then click [Browse] > select .json files (up to 10) > [Load Intent].

Analyze and Generate Model

Please continue from Quickstart Guide Step 4 to complete the process.

Using Loki Templates

Loki currently provides templates in Python and Java (Android). More programming languages will be supported in the future.

Compile Calculation Logic for Project Intents

Loki Template Directory Structure

main.py
intent/Loki_Intent.py
intent/Loki_xxxxx.py

Go back to the Loki homepage and click on the desired programming language (e.g., Python) under [Project Template] to download it.

Unzip the .zip file. The file structure will look like this:
In the intent/Loki_Intent.py file, begin writing the intent semantic computation logic.
Import the project and start using Loki。

Configure Large Language Models (LLM)

Default llm.json

{
    "[LiteLLM] OpenAI ChatGPT": {
        "type": "litellm",
        "model": "gpt-4o",
        "env": {
            "OPENAI_API_KEY": "${API_KEY}"
        }
    },
    "[LiteLLM] Azure ChatGPT": {
        "type": "litellm",
        "model": "azure/${DEPLOYMENT_NAME}",
        "env": {
            "AZURE_API_KEY": "${API_KEY}",
            "AZURE_API_BASE": "${ENDPOINT}",
            "AZURE_API_VERSION": "2023-05-15"
        }
    },
    "[LiteLLM] Google Gemini": {
        "type": "litellm",
        "model": "gemini/gemini-pro",
        "env": {
            "GEMINI_API_KEY": "${API_KEY}"
        }
    },
    "[LiteLLM] Anthropic Claude": {
        "type": "litellm",
        "model": "claude-opus-4-20250514",
        "env": {
            "ANTHROPIC_API_KEY": "${API_KEY}"
        }
    },
    "[LiteLLM] AWS Sagemaker": {
        "type": "litellm",
        "model": "sagemaker/${ENDPOINT_NAME}",
        "env": {
            "AWS_ACCESS_KEY_ID": "",
            "AWS_SECRET_ACCESS_KEY": "",
            "AWS_REGION_NAME": ""
        }
    },
    "[LiteLLM] Ollama": {
        "type": "litellm",
        "url": "http://localhost:11434",
        "model": "ollama/llama2"
    },
    "[API] Google Gemini": {
        "type": "api",
        "url": "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash:generateContent?key=${API_KEY}",
        "header": {
            "Content-Type": "application/json"
        },
        "payload": {
            "contents": [{
                "parts": [{
                    "text": ""
                }]
            }],
        }
    },
    "[API] Anthropic Claude": {
        "type": "api",
        "url": "https://api.anthropic.com/v1/messages",
        "header": {
            "x-api-key": "${API_KEY}",
            "anthropic-version": "2023-06-01",
            "content-type": "application/json"
        },
        "payload": {
            "model": "claude-3-7-sonnet-20250219",
            "max_tokens": 1024,
            "messages": ""
        }
    }
}

Setting Up LLM

To set or customize an LLM, first open the llm.json，file. If the file does not exist, simply run LokiTool once—this will automatically generate the default llm.json。

The system supports two methods for connecting to an LLM: LiteLLM and REST API. Be sure to replace ${VAR} with the appropriate values; otherwise, the model cannot be loaded.

LiteLLM

Supports common models available on the market. Please refer to the LiteLLM documentation to configure the model and the env variables.
REST API

The API can connect to a customized LLM server, but it may not work if the request and response formats differ. The configuration must include the following fields: url, header, and payload.
- url should be a string
- header and payload should be dictionaries
- The payload must contain the string

When the system runs preview_run or run_alias, it will read the selected LLM configuration and replace the string in the payload with the value of data.messages.

Calling Call API

Sample Code 1:

from requests import post

url = "http://LokiTool_URL/loki_EN/call/"
payload = {
    "project": "ProjectName",
    "intent": "IntentName",
    "func": "insert_utterance",
    "data": {
        "utterance": ["How is the weather today"],
        "checked_list": ["ENTITY_noun"]
    }
}

response = post(url, json=payload)

var url = "http://LokiTool_URL/loki_EN/call/";

var xhr = new XMLHttpRequest();
xhr.open("POST", url);

xhr.setRequestHeader("Accept", "application/json");
xhr.setRequestHeader("Content-Type", "application/json");

xhr.onreadystatechange = function () {
    if (xhr.readyState === 4) {
        console.log(xhr.status);
        console.log(xhr.responseText);
    }
};

var data = `{
    "project": "ProjectName",
    "intent": "IntentName",
    "func": "insert_utterance",
    "data": {
        "utterance": ["How is the weather today"],
        "checked_list": ["ENTITY_noun"]
    }
}`;

xhr.send(data);

Result returned as a JSON object:

{
    "status": true,
    "msg": "Success!",
    "result_list": [{"status": true, "msg": "Success!"}]
}

Sample Code 2:

from requests import post

url = "http://LokiTool_URL/loki_EN/call/"
payload = {
    "project": "ProjectName",
    "intent": "IntentName",
    "func": "insert_assistant",
    "data": {"assistant": [
        {"title": "Heavy Rain", "content": {"DEFINE": "Rainfall where the accumulated rainfall in 24 hours reaches 80 mm or more, or the hourly rainfall reaches 40 mm or more."}},
        {"title": "Torrential Rain", "content": {"DEFINE": "Rainfall where the accumulated rainfall in 24 hours reaches 200 mm or more, or the accumulated rainfall in 3 hours reaches 100 mm or more."}},
        {"title": "Extremely Torrential Rain", "content": {"DEFINE": "Rainfall where the accumulated rainfall in 24 hours reaches 350 mm or more, or the accumulated rainfall in 3 hours reaches 200 mm or more."}},
        {"title": "Catastrophic Rain", "content": {"DEFINE": "Rainfall where the accumulated rainfall in 24 hours reaches 500 mm or more."}}
    ]}
}

response = post(url, json=payload)

var url = "http://LokiTool_URL/loki_EN/call/";

var xhr = new XMLHttpRequest();
xhr.open("POST", url);

xhr.setRequestHeader("Accept", "application/json");
xhr.setRequestHeader("Content-Type", "application/json");

xhr.onreadystatechange = function () {
    if (xhr.readyState === 4) {
        console.log(xhr.status);
        console.log(xhr.responseText);
    }
};

var data = `{
    "project": "ProjectName",
    "intent": "IntentName",
    "func": "insert_assistant",
    "data": {"assistant": [
        {"title": "Heavy Rain", "content": {"DEFINE": "Rainfall where the accumulated rainfall in 24 hours reaches 80 mm or more, or the hourly rainfall reaches 40 mm or more."}},
        {"title": "Torrential Rain", "content": {"DEFINE": "Rainfall where the accumulated rainfall in 24 hours reaches 200 mm or more, or the accumulated rainfall in 3 hours reaches 100 mm or more."}},
        {"title": "Extremely Torrential Rain", "content": {"DEFINE": "Rainfall where the accumulated rainfall in 24 hours reaches 350 mm or more, or the accumulated rainfall in 3 hours reaches 200 mm or more."}},
        {"title": "Catastrophic Rain", "content": {"DEFINE": "Rainfall where the accumulated rainfall in 24 hours reaches 500 mm or more."}}
    ]}
}`;

xhr.send(data);

Result returned as a JSON object:

{
    "status": true, 
    "msg": "Success!"
}

Use Loki Call to perform various Loki-related tasks.

HTTP Request

POST http://LokiTool_URL/loki_EN/call/

Arguments

Arguments	Type	Default	Description
project	str	""	Name of the project. If it does not exist, the system will create it automatically.
intent	str	""	Name of the intent. If it does not exist, it will be created automatically.
func	str	""	Can be set to one of the following options:
			- create_project: Create a new project.
			- create_intent: Create a new intent.
			- get_info: Retrieve project information.
			- get_utterance: Retrieve all utterances within an intent.
			- insert_assistant: Add a large amount of ChatGPT Assistant content.
			- insert_utterance: Add a large number of utterances for an intent.`Note: Utterances with the same structure will not be added.`
			- match_utterance: Check if a sentence has the same structure as one in the intent.
			- preview_alias: Preview the content with applied aliases.
			- reset_alias: Clear all prompt aliases.
			- reset_userdefined: Clear the user-defined dictionary.
			- run_alias: Apply aliases and run the ChatGPT API.
			- set_llm: Set the LLM for the project.
			- update_alias: Update prompt aliases.`Existing aliases will not be removed.`
			- update_prompt: Update the ChatGPT prompt settings.
			- update_userdefined: Update the user-defined dictionary.`Existing dictionary entries will not be removed.`
data	dict	""	Set the following formats based on the func parameter
data.alias	dict	{}	Prompt alias settings to be updated. `{text: {key: value_list}, regex: {key: value_list},` `ner: {id: false, person: false, address: false, route: false, url: false}}` func: `update_alias`
data.assistant	dict	{}	ChatGPT Assistant content to be added.`{title: value, content: {key: value}}` func: `insert_assistant`
data.checked_list	str[]	[]	POS tags to be pre-selected when adding new utterances (refer to POS tags)。 func: `insert_utterance`
data.language	str	"en-us"	The language used in the project can only be `en-us`. func: `create_project`
data.llm	str	""	Sets the generative model to be used by the project. func: `set_llm`
data.messages	dict	{}	Messages parameter for executing the ChatGPT API.`[{role: user, content: value}]` func: `preview_alias` `run_alias`
data.name	str	""	Project name. func: `create_project`
data.prompt	dict	{}	ChatGPT Prompt settings to be updated.`{system: value, assistant: value, user: value}` func: `update_prompt`
data.type	str	"intent" "basic"	Model type for the project must be `intent`. func: `create_project` Type of intent, must be either `basic` or `advance`. func: `create_intent`
data.user_defined	dict	{}	user-defined dictionary to be updated. `{key: value_list}` func: `update_userdefined`
data.utterance	str[]	[]	Sentences to be analyzed. func: `insert_utterance` `match_utterance`

Returned message

Returned message	Type	Description
status	bool	The value cloud be True or False。
msg	str	The value could be as listed below:
		- Success!: Task executed successfully.
		- Failure.: Task execution failed.
		- Internal server error. (Your word count balance is not consumed, don't worry. System will reboot in 5min, please try again later.): Well...it seems there's something wrong with our server. Don't worry, we are fixing it and the server is scheculed to reboo and go online in 5 min. Please try again later. Don't worry, your quota stays intact.
result	any	Shows the result for `preview_alias` and `run_alias`.
result_list	list	Shows detailed results for `insert_utterance`, `match_utterance`, `get_utterance`.
alias_setting	dict	Shows the Prompt alias settings applied in `preview_alias`.

Calling Command API (Consider using Loki Call instead.)

Sample Code:

from requests import post

url = "http://LokiTool_URL/loki_EN/command/"
payload = {
    "project": "ProjectName",
    "intent": "IntentName",
    "type": "insert_utterance", # basic / advance
    "utterance": ["sentence"]
}

response = post(url, json=payload)

var url = "http://LokiTool_URL/loki_EN/command/";

var xhr = new XMLHttpRequest();
xhr.open("POST", url);

xhr.setRequestHeader("Accept", "application/json");
xhr.setRequestHeader("Content-Type", "application/json");

xhr.onreadystatechange = function () {
    if (xhr.readyState === 4) {
        console.log(xhr.status);
        console.log(xhr.responseText);
    }
};

var data = `{
    "project": "ProjectName",
    "intent": "IntentName",
    "type": "IntentType", // basic / advance
    "utterance": ["sentence"]
}`;

xhr.send(data);

Result returned as a JSON object:

{
    "status": true,
    "msg": "Success!",
    "result_list": [{"status": true, "msg": "Success!"}]
}

Use Loki Command to quickly and massively add or match sentences for intents.

HTTP Request

POST http://LokiTool_URL/loki_EN/command/

Arguments

Arguments	Type	Default	Description
project	str	""	Name of the project. If it does not exist, the system will create it automatically.
intent	str	""	Name of the intent. If it does not exist, it will be created automatically.
type	str	""	Type of intent, must be either `basic` or `advance`.
utterance	str	""	A single sentence to be analyzed and added.
	list	[]	Multiple sentences to be analyzed and added.

Returned message

Returned message	Type	Description
status	bool	The value cloud be True or False。
msg	str	The value could be as listed below:
		- Success!: Intent detection completed successfully.
		- Project create success!: Project created successfully.
		- Intent create success!: Intent created successfully.
		- Project already exist!: A project with the same name already exists.
		- Intent already exist!: An intent with the same name already exists within the project.
		- Utterance already exist.: The sentence already exists in the intent.
		- Pattern same as Utterance.: A sentence with the same structure already exists in the intent.
		- Internal server error. (Your word count balance is not consumed, don't worry. System will reboot in 5min, please try again later.): Well...it seems there's something wrong with our server. Don't worry, we are fixing it and the server is scheculed to reboo and go online in 5 min. Please try again later. Don't worry, your quota stays intact.
result_list	list	The results of sentence creation.