RedPen 1.8 Documentation

RedPen requires the following software.

First, download the RedPen package from release page, and then decompress the package with the following commands.

Then, run the redpen command with the supplied sample document and configuration file.

RedPen provides a simple command line tool called 'redpen' to check documents.

RedPen also provides the server. RedPen server provides not only UI but also practical REST API (see RedPen Server section). The following is an image of RedPen server UI.

Plain text output format consists of the following lines.

An alternate plain text form (plain2) prints each sentence, followed by all of the errors found in the sentence.

The top section of the XML output format is validation-result element which contains multiple error sections. Each error section has the following sub-elements.

The alternative JSON output format (json2) collates each error by the sentence it relates to.

SentenceLength validator checks the length of sentences in the input document. If the length of the sentence is greater than the specified maximum length, the validator generates a warning.

InvalidExpression validator checks if input sentences contain invalid expressions (words or phrases). If the input sentence contains invalid expressions, the validator generates a warning.

InvalidWord validator checks if input sentences contain invalid words. If the input sentence contains invalid words, the validator generates a warning.

This validator checks if there is a white space at the end of sentences (except for the last sentence of paragraph). If the input sentence does end with a white space, a warning is given.

CommaNumber validator checks the number of commas in a sentence.

WordNumber validator checks the number of words in one setnece.

SuggestExpression validator works in a similar way to the InvalidExpression validator. If the input sentence contains invalid expressions, this validator returns a warning suggesting the correct expression.

Some symbols or characters have alternate characters with the same role. For example question mark ? (0x003F) has another unicode variation ？(0xFF1F). InvalidSymbol checks if input sentences contains invalid characters or symbols. The symbols settings are added into the character setting block int the configuration file. In this file, we write the symbols we should use in the document and their invalid counterparts. The details of these settings is described in the next section.

Some symbols need space before or after them. For example, if we want to ensure a space before a left parentheses, we can add the preference to the symbol setting block (see Setting symbols)

KatakanaEndHyphen validator checks the end hyphens of Katakana words in Japanese documents. Japanese Katakana words have variations in their end hyphen. For example, "computer" is written in Katakana as "コンピュータ" (without hyphen), and "コンピューター" (with hypen). This validator checks to ensure that Katakana words match the predefined standard. See JIS Z8301, G.6.2.2 b) G.3.

KatakanaSpellCheck validator checks if Katakana words have variational written form. For example, if the Katakana word "インデックス" and the variational form "インデクス" exist within the same document, this validator will return a warning.

SectionLength validator checks the maximum number of words allowed in an section.

ParagraphNumber validator checks the maximum number of paragraphs allowed in one section.

ParagraphStartWith validator checks to see if the characters at the beginning of paragraphs conforms to the correct style.

SpaceBetweenAlphabeticalWord validator checks that alphabetic words are surrounded with whitespace. This validator is used in non-latin languages such as Japanese or Chinese.

Contraction throws an error when contractions are used in a document in which more than half of the verbs are written in non-contracted form.

Spelling validator throws an error if there are spelling mistakes in the input documents. This validator only works for English documents.

DoubledWord validator throws an error if a word is used more than once in a sentence. For example, if an input document contains the following sentence, the validator will report an error since good is used twice.

DoubledJoshi throws an error if a Joshi (Japanese part-of-speech) is used more than once in a Japanese sentence.

SuccessiveWord validator throws an error if the same word is used twice in succession. For example, if an input document contains the following sentence, the validator will report an error since is is used twice in succession.

DuplicatedSection validator throws an error if there are section pairs which have almost the same content.

JapaneseStyle validator reports errors if the input file contains both "dearu" and "desu-masu" style.

DoubleNegative validator reports errors when input sentence contains double negative expression.

This validator reports an error if too many sentences start with the same sequence of words.

This validator ensures that there are candidates for expanded versions of acronyms somewhere in the document.

That is, if there exists an acronym ABC in the document, then there must also exist a sequence of capitalized words such as Axxx Bxx Cxxx.

This validator ensures that usage of specific words in the document don’t occur too frequently. It calculates the frequency that words are used and compares them the a reference histogram of word frequency for written English.

Excessive deviation from normal usage generates a validation error.

This validator ensures that sequences of words that are hyphenated in the dictionary are hyphenated in your document.

This validator ensures that numbers in a sentence are formatted using commas (ie: 12,000 instead of 120000), and don’t have excessive decimal points.

This validator generates errors if parenthesized sentences (such as this) are used too frequently, or are nested too heavily.

This validator generates errors if sequences of words form what is generally considered to be a "weak expression."

This validator generates errors if the style end of sentence is American style.

This validator generates errors if the Hankaku Kana characters are used in input document.

This validator generates errors if input sentence uses invalid Okurigana Style (Japanese).

This validator generates errors if input sentence start with a capital character.

This validator generates errors if sections in input documents do not have any paragraphs or sentences.

This validator generates errors if sections in input documents do not have any paragraphs or sentences.

This validator generates errors when the level of child sections (chapters) has the gap. For example, The following is a misplaced section sample.

In the above example, chapter 1 should have section 1.1 before subsection 1.1.1.

This validator generates errors when input sentences has a words consist of too many Kanji characters.

In the above example, chapter 1 should have section 1.1 before subsection 1.1.1.

This validator generates errors when input documents contains smaller sections than specified.

This validator generates errors when Japanese documents contains the ambiguous noun conjunction pattern. The ambigous pattern is that two nouns are conjuncted with Joshi, no (の).

The following is a sample of this pattern.

JapaneseNumberExpression checks if the number expressions in the input text are in the consistent style.

SuccessiveSentence throws an error when it find almost the same sentences are in succession. This validator is useful to check the human error as follows.

In the above sample, the same sentences are used in succession. This is a human error.

RedPen has a single configuration file, which contains all the settings RedPen requires. The main configuration file is an xml file with a root element of redpen-conf. Within this element there are two sub elements named "validators" and "symbols."

To apply the default validators and symbol settings for a target language, we just specify lang attributes in the redpen-conf element.

The validators section specifies which validators are to be loaded by RedPen. Each validator within this section can have their property values overridden.

The symbols section overrides the default symbol settings for the target language. The following is an example of a RedPen configuration file.

In the next section we will cover the configuration of validators in greater detail. The settings for the symbols section are described in the Setting Symbols section.

The RedPen configuration file contains a "validators" section for registering Validators. RedPen will apply each validator specified in this section to the to the input document.

The following is a sample "validators" section.

Each validator is configured within its own validator element. The name attribute of this element specifies the name of the function (called validator). Each validator is responsible for checking a particular aspect of the input document. For example, if the "SectionLength" is added in the configuration, then RedPen will check the length of each section in the input document.

Some validator components can be configured using "property" elements. For example, you can override the maximum character count in the "SectionLength" validator by specifying a "max_num" property. Some validators also have "sub-validators" which can also be configured within the validator section.

We cover all supported validators in the Validator section.

The lang attribute of the redpen-conf element determines how various symbols are handled by RedPen. RedPen supports default symbols for "en" and "ja", which are described in English symbols and Japanese Symbols.

The default symbol settings for a target language can be overridden by configuring the symbols section of the RedPen configuration file.

The default settings are described in the following sections. Within the symbols configuration section we can use symbol elements to specify which symbols to use when validating documents. Each "symbol" element overrides a character found in the documents.

The following table describes the properties of the symbol element.

Plain text supports a set of paragraphs. Paragraphs are separated by two new lines. For example, the following article has two paragraphs.

See the AsciiDoctor reference

RedPen supports a subset of Wiki syntax. Currently, the supported elements of Wiki syntax are as follows.

RedPen currently supports the following Markdown elements.

See the Re:VIEW reference

Properties files or Resource Bundles are commonly used for internalization in Java. RedPen treats every property as a section, which can have one or more sentences. Comments and values, but not keys are validated.

See the Properties Javadoc for more information on file format.

For AsciiDoc text, writers add the suppress annotation in attribute block. The annotation is [suppress]. For example, the following AsciiDoc text suppresses the all the erorrs in the section.

When the writers want to suppress only the specified errors, they add Validator names after suppress. For example, the following example suppresses only two types of errors (Contraction WeakExpression).

For Markdown text, writers add the suppress annotation in HTML comments. Specifically Writers add annotation, @suppress in HTML comments. For example, the following text suppress all the error in the section.

To suppress only specific errors, writers add the Validator names following @suppress annotation. The following sample suppresses the two type errors (Contraction WeakExpression).

For Re:VIEW text, writers add the suppress annotation in comments. Specifically Writers add annotation, @suppress in Re:VIEW comment #@#. For example, the following text suppress all the error in the section.

To suppress only specific errors, writers add the Validator names following @suppress annotation. The following sample suppresses the two type errors (Contraction WeakExpression).

For LaTeX texts, writers can add Markdown style annotation in LaTeX comment (%).

RedPen users can extend RedPen by creating new Validators. This section describes how we construct Validators and covers the basics of the internal document model used by Validators.

RedPen users can create new validators for themselves or their organization. Adding validator is simple - just write a class which extends the abstract class Validator.

For those who are unfamiliar with Java, RedPen version 1.3 introduced JavaScriptValidator, which is a special validator that loads Validator implementations written in JavaScript.

This section describes the internal document model structure generated by parser objects.

Generated RedPen documents consist of several blocks, which represent the elements of a document.

The following image shows the document model used by RedPen.

Currently there are two language settings, en, ja and ru. en is for documents written in Latin-based languages such as English and German. "ja" is for Japanese documents.

In order to override the language, you must change the "lang" property of the "symbols" section of the configuration file. In the following example, the config file specifies Japanese ("ja") as the document language.

Depending on the documents and their authors, the characters and symbols used may differ. For example, one author prefers "'" for a left single quotation mark, whereas another author prefers to use "‘" for a left single quotation mark.

For such cases, RedPen provides the way to override the default symbols (see Setting Symbols section) used in the document. In addition, specify which symbols should not used in the input documents.

The following overrides the setting for single quotation marks to enforce the use of the ascii quotation mark (').

Please refer to the Commands section for details on how-to start the RedPen server.

RedPen server is also launch in the Heroku environment with few clicks. In the bottom of RedPen README, there is a Heroku Button.

Clicking the buttom, a RedPen server get started in Heroku. Actually the RedPen sample server in the RedPen home page is working in Heroku environment.

To build RedPen, you need to install the following tools.

We will develop this tiny tool slowly but steadily. Therefore we would not be able to accept the proposal containing drastic changes, even when the proposed change is useful for the project.

Building all the RedPen package is simple. You just run the following command.

The built RedPen package is flushed in redpen/redpen-distribution/target. After you get the built RedPen package, you unzip it with the following command.

Now the RedPen package is ready to use.

When you modify RedPen source files, please make sure to add the test cases to make the reviewer understand the expected behaviors. In addition run whole test cases before making the Pull Request. RedPen test cases are run with the following command.

https://twitter.com/redpen_news/

https://github.com/redpen-cc/redpen

https://gitter.im/redpen-cc/redpen

https://github.com/redpen-cc/redpen/issues

Atom is a modern text editor which works in various platforms such as OS X, Linux, and Windows. linter-redpen is an sophisticated Atom package created by griffin-stewie. This provides real-time checking of texts.

Emacs is a real-time display text editor. Many software engineers love this editor for the customisablity. redpen-paragraph is a Emacs package for RedPen created by karronoli.

A unite.vim integration of redpen for automatic proofreading is created by rhysd

IntelliJ IDEA is a popular Java IDE (Integrated Development Environment) provided Jetbrains. But this IDE also provides many plugins to support editing markup texts such as AsiiDoc or Markdown. We have been developing IntelliJ IDEA plugin.