--- /dev/null
+%#!lualatex ajt-devel-ltja
+\documentclass{ajt}
+
+%%% Packages used in this paper
+%%% LuaTeX-ja
+
+\usepackage{luatexja}
+\DeclareFontShape{JY3}{mc}{m}{n}{<-> s*[0.92489] file:ipaexm.ttf:jfm=ujis}{}
+\DeclareFontShape{JY3}{gt}{m}{n}{<-> s*[0.92489] file:ipaexg.ttf:jfm=ujis}{}
+
+%%% for LTXexample environment
+\usepackage{showexpl,lltjlisting}
+\lstset{basicstyle=\ttfamily, width=0.3\textwidth}
+
+%%% for logo
+\usepackage{mflogo}
+
+%%% Verbatim environment
+\usepackage{fancyvrb}
+\CustomVerbatimEnvironment{code}{Verbatim}%
+{numbers=left,xleftmargin=1.5em,baselinestretch=1.069,fontsize=\small}
+\CustomVerbatimEnvironment{codewithoutnum}{Verbatim}%
+{xleftmargin=1.5em,baselinestretch=1.069,fontsize=\small}
+\CustomVerbatimEnvironment{codewithoutnumsmall}{Verbatim}%
+{xleftmargin=1.5em,baselinestretch=1.0,fontsize=\footnotesize}
+\DefineShortVerb{\|}
+
+%%% Mandatory article metadata %%%
+\title{The development of \LuaTeX-ja package}
+\author{Hironori Kitagawa}
+\address{The \LuaTeX-ja project team}
+\email{h\_kitagawa2001@yahoo.co.jp}
+
+\keywords{\TeX, p\TeX, \LuaTeX, \LuaTeX-ja, Japanese}
+\abstract{%
+The \LuaTeX-ja package is a macro package for typesetting Japanese documents under \LuaTeX.
+This packages has much flexibility of typesetting than p\TeX, and corrected some unwanted features of p\TeX.
+In this paper, we describe specifications, the current status and some internal processing codes of \LuaTeX-ja.
+}
+
+\newcommand{\parname}[1]{\textsf{#1}}
+
+\begin{document}
+
+%%% Do not forget to start with \maketitle!
+\maketitle
+
+\section{Introduction}
+\subsection{History}
+To typeset Japanese documents with \TeX, ASCII p\TeX~\cite{ptex} has
+been widely used. There are other methods---for example, using Omega
+and OTP~\cite{omegaj}, or with the CJK package---to do so, however,
+these alternative methods did not became a majority. On the one hand,
+p\TeX\ enables us to produce high-quality documents, but on the other
+hand, p\TeX\ is left behind from the extensions of \TeX\ such as \eTeX\
+and \pdfTeX, and the diffusion of UTF-8 encoding. In recent years, the
+situation become better, because of the developments of |ptexenc|~\cite{ptexenc} by Nobuyuki Tsuchimura,
+$\varepsilon$-p\TeX~\cite{eptex} by the author,~and
+up\TeX~\cite{uptex} by Takuji Tanaka.
+
+However, there are still lag now.
+
+
+Before this \LuaTeX-ja package, there were several attempts to typeset Japanese documents under \LuaTeX.
+Here we cite three examples:
+\begin{itemize}
+\item |luaums.sty|~\cite{luaums} developed by the author. This experimental package is for creating a Japanese-based presentation under \LuaTeX.
+\item |luajalayout| package\cite{luajalayout}, formerly known as the |jafontspec| package, by Kazuki Maeda.
+This package is based on \LaTeXe\ and |fontspec| package.
+\item |luajp-test| package\cite{luajp-test}, a test package made by Atsuhito Kohda, based on articles on the web page~\cite{joylua}.
+\end{itemize}
+% ltja: hassan boushi
+
+\subsection{Development Policy of \LuaTeX-ja}
+\label{ssec-pol}
+The first aim of the project is to implement features of p\TeX as macros under \LuaTeX, so \LuaTeX-ja is much affected by p\TeX.
+However, as the development proceeds, some technical/conceptual difficulties are arised. Hence we changed the aim of the project:
+\begin{itemize}
+\item\emph{\LuaTeX-ja offers more flexibility of typesetting than that by
+ p\TeX.}
+
+ There is JIS~X~4051 Japanese Standard, ...
+ However, we think that the ability of producing outputs conformed to JIS~X~4051 is not enough for (Japanese) typesetting;
+ the \TeX\ engine can handle the case even when one wants to produce very incoherent output for some reason.
+ In this point, we can say p\TeX has some flexibility, hence \LuaTeX-ja should have more.
+
+ For this reason, \LuaTeX-ja has counterparts of additional primitives of p\TeX.
+
+\item\emph{\LuaTeX-ja isn't mere re-implementation or porting of p\TeX;
+ some (technically and/or conceptually) inconvenient features of
+ p\TeX\ are modified.}
+
+ We describe this point in more detail at the next section.
+\end{itemize}
+
+
+\subsection{Contents of this Paper}
+Here we describe the contents of the rest of this paper briefly.
+In Section~2, we describe major differences between p\TeX\ and \LuaTeX-ja,
+which is introduced. Some of them are due to specifications of callbacks
+in \LuaTeX\ (\emph{i.e.}, technical reason), and others are which we
+thought which are better to be changed, for ``natural''
+specifications. In Section~3, we show the current status of the \LuaTeX-ja project.
+
+For implementing features into \LuaTeX-ja, we had to use some tricks in Lua scripts.
+In Section~4, we describe several these tricks and internal processing methods.
+We hope that the materials in this section, espcially
+Subsection~4.3, have good applications.
+
+
+\section{Major differences with \pTeX}
+In this section, we breifly look at ** major differences between p\TeX\ and \LuaTeX-ja.
+
+\subsection{Names of Control Sequences}
+Since p\TeX\ is a modification of a \TeX\ engine, some primitives added in it takes a form that cannot be simulated by a macro.
+For example, an additional primitive |\prebreakpenalty|$\langle\hbox{\it
+char\_code}\rangle$|=|$\langle\hbox{\it penalry}\rangle$ in p\TeX\ sets the
+amount of penalty inserted before $\langle\hbox{\it char\_code}\rangle$
+to $\langle\hbox{\it penalry}\rangle$.
+
+Moreover, there are some parameters for Japanese typesetting which were
+mere internal integers, dimensions, or~skips in p\TeX\ that cannot be
+implemented by same approaches in \LuaTeX-ja. These parameters have a
+common feature; the values at the end of a horizontal box or that of a
+paragraph are effective in whole box or apragraph. A good example of
+them is |\kanjiskip|, the default amount of a skip which will be
+inserted between two consecutive Japanese characters by default. The
+reason of this is the place of |hpack_filter| in the \LuaTeX's
+CWEB-source code, and we will discuss on it in
+Subsection~\ref{ssec-stack}.
+
+From above 2~problems we discussed above, the assignment and retrieval
+of most parameters in \LuaTeX-ja are summarize into 3~control sequences:
+\begin{itemize}
+\item |\ltjsetparameter{|$\langle\hbox{\it
+ name}\rangle$|=|$\langle\hbox{\it value}\rangle$|,...}|: for local
+ assignment.
+\item |\ltjglobalsetparameter|: for global assignment.
+\item |\ltjgetparameter{|$\langle\hbox{\it
+ name}\rangle$|}[{|$\langle\hbox{\it optional
+ argument}\rangle$|}]|: for retrieval. The returned value is always
+ a string.
+\end{itemize}
+
+\subsection{Linebreak after Japanese Character}
+\label{ssec-line}
+Japanese texts can linebreak almost everywhere, in contrast with
+alphabetic texts can linebreak only between on words (or use
+hyphenation). Hence, p\TeX's input processor is modified so that a
+linebreak after a Japanese character doesn't emit a space. However,
+there is no way to customize the input processor of \LuaTeX. All we can
+do is that modify an input line before when \LuaTeX\ begin to process
+it, using |process_input_buffer| callback.
+
+Hence, in \LuaTeX-ja, a comment letter (we reserve U+FFFFF for this
+purpose) will be appended to an input line, if this ends with a Japanese
+character\footnote{Strictly speaking, it also requires that the catcode
+of the endline character is 5.}. One might jump to a conclusion that the
+treatment of linebreaks by p\TeX and that of \LuaTeX-ja is very similar,
+but they are different in the respect that \LuaTeX-ja's judgement
+whether a comment letter will be appended the line is done \emph{before}
+the line is processed by \LuaTeX.
+
+Figure~\ref{fig-linebreak} shows an example; the command at the first
+line marks most of Japanese characters as ``non-Japanese character''. In
+other words, from this command onward, the letter `あ' will not be
+received any process of \LuaTeX-ja. Then, it is natural to occur a space
+between `あ' and `y' in the output, where the actual output does not so.
+This is because `あ' is considered to be a Japanese character by
+\LuaTeX-ja, when \LuaTeX-ja look the end of the input line~2.
+\begin{figure}
+\begin{LTXexample}
+\font\x=IPAMincho \x
+\ltjsetparameter{jacharrange={-6}}xあ
+y
+\end{LTXexample}
+\caption{A notable sample showing the treatment of a linebreak after a Japanese character.}\label{fig-linebreak}
+\end{figure}
+
+\subsection{Separation between ``real'' fonts and Metrics}
+Traditionally, most Japanese fonts used in typesetting are monospaced,
+that is, most glyphs have same size and square-shaped. Hence, it is not
+rare that the contents of different Japanese TFMs (JFM, for short) are
+totally same, and only differ in their names. For example, the
+difference between |min10.tfm| and |goth10.tfm|, which are JFMs shipped
+with p\TeX\ for \emph{Mincho} and \emph{Gothic} fonts, are their
+|FAMILY| and |FACE| only. Another example is: if one want to use many
+fonts which are not installed in his \TeX\ distribution, of course he
+needs to prepare TFMs for them. But, as long as he wants to use Japanese
+fonts with p\TeX, he has to only copy and rename some JFM (\emph{e.g.,}~copy |jis.tfm| to |hoge.tfm|).
+
+Considering this situation, we decided to separate ``real'' fonts and
+Metrics in \LuaTeX-ja, as shown in Figure~\ref{fig-jfdef};
+\begin{itemize}
+\item a control sequence |\jfont| must be used for japanese fonts, instead of |\font|.
+\item \LuaTeX-ja automatically loads the |luaotfload| package, so
+ |file:| prefix and features can be used as the line~1 in
+ Figure~\ref{fig-jfdef}.
+\item The |jfm| key specifies the metric for the font. In
+ Figure~\ref{fig-jfdef}, both fonts will use a metric stored in a Lua script named
+ |jfm-ujis.lua|.
+\item The |psft:| prefix can be used to specify name-only, noembedded
+ fonts.
+\end{itemize}
+We note that |-kern| in features is important, since if kerning information from real font itself will clash with spacing from the metric.
+
+\begin{figure}
+\begin{verbatim}
+\jfont\foo=file:ipaexm.ttf:jfm=ujis;script=latn;-kern;+jp04 at 12pt
+\jfont\bar=psft:Ryumin-Light:jfm=ujis at 10pt
+\end{verbatim}
+\caption{Typical declarations of Japanese fonts}
+\label{fig-jfdef}
+\end{figure}
+
+\subsection{Insertion of Kerns and/or Glues for Japanese Typesetting: The Timing}
+As described in \cite{luatexref}, \LuaTeX's kerning and ligaturing
+process is totally different from that of Knuth's original \TeX82.
+\TeX82's process is done just when a (sequence of) character node is
+appended to current list, hence we can cancel this process by writing
+|f{}irm| (this gives `f\hbox{}irm' in \TeX82). However, \LuaTeX's process is
+\emph{node-based}, that is, the process will be done when a horizontal
+box of a paragraph is ended, so |f{}irm| and |firm| yield the same
+output under \LuaTeX.
+
+The situation for Japanese characters is basically same, but not entirely.
+Glues (and kerns) those will be needed for Japanese typesetting will be divided into the following three categories:
+\begin{description}
+\item[Glues (and Kerns) from the Metric of Japanese Fonts]
+\item[Default Skip Between a Japanese Character and an Alphabetic Character]
+Usually 1/4 of fullwidth with some stretch and shrink.
+\item[Default Skip Between Two Consecutive Japanese Characters]
+\end{description}
+In p\TeX, these three kinds of glues are treated differently. The first
+ category (\emph{JFM glue}, for short) is inserted when a Japanese
+ character node is appended to current list, same as alphabetic
+ characters in \TeX82. The second category (\emph{xkanjiskip}, for
+ short) is inserted just before `hpack' or line-breaking of a paragraph;
+ this timing is somewhat similar to that of \LuaTeX's kerning
+ process. The third category (\emph{kanjiskip}, for short) is not
+ appeared as a node anywhere; only appears implicitly in calculation of
+ width of a horizontal box of line-breaking. These specifications made
+ p\TeX's behavior very hard to understand.
+
+\LuaTeX-ja does the insertion process of glues in all three categories
+simultaneously, inside |hpack_filter| and |pre_linebreak_filter|
+callbacks. The reasons of this specification are to behave like
+alphabetic characters in \LuaTeX\ (as described in the first paragraph),
+and to clarify the specification for \LuaTeX-ja's process.
+
+
+\subsection{Insertion of Kerns and/or Glues for Japanese Typesetting: The Behavior}
+
+
+
+\section{Current Status of the Development}
+\LuaTeX-ja
+
+\section{Implementation}
+\subsection{Handling of Japanese Fonts}
+In p\TeX, there are three slots for maintaining current fonts, namely
+|\font| for alphabetic fonts, |\jfont| for Japanese font (in horizontal
+direction) and |\tfont| for Japanese font (in vertical direction). With these slots, we can select
+current font for alphabetic characters and that for Japanese characters separately in p\TeX.
+However, \LuaTeX\ has only one slot for maintaining current font, as \TeX82.
+This situation leads a problem: how can we maintain ``current Japanese font''?
+
+There are three approaches for this problem. One approach is to make a
+mapping table from alphabetic fonts to corresponding Japanese fonts
+(here we don't assume that NFSS2 is available), and when current
+alphabetic font is changed, current Japanese font also changes according
+to the table. Another approach is that we always use composite fonts
+with alphabetic fonts and Japanese fonts. The third approach is that the
+information of current Japanese font is stored in an attribute. We
+adopted the third approach, since \LuaTeX-ja is much affected by p\TeX\
+as we noted in Subsection~\ref{ssec-pol}.
+
+
+
+
+\subsection{Overview of the Processes}
+Now we describe an outline of the \LuaTeX-ja's process briefly.
+\begin{description}
+\item[Treatment of Linebreaks after Japanese Characters] We described
+ this already at Subsection~\ref{ssec-line}. Registered in the
+ |process_input_buffer| callback.
+\item[Font Replacement] In the |hyphenate| callback, we looks into for
+ each \textit{glyph\_node}~$p$. If its character is considered
+ to be a Japanese character, the font used in $p$ is replaced
+ by the value of |\ltj@curjfnt| that is associated
+ with~$p$. Also we subtract the subtype of $p$ by 1 to
+ suppress hyphenation around it by \LuaTeX, since later
+ processes of \LuaTeX-ja take care of all things about
+ Japanese charaters.
+\end{description}
+%
+Following processes are all executed in |pre_linebreak_filter| and |hpack_filter| callback. These are main routines of \LuaTeX-ja:
+
+\begin{description}
+\item[Examination of Stack Level] We traverse the horizontal list which is the content of a horizontal box
+to determine what is the level of \LuaTeX-ja's internal stack in the end
+ of the list. This is needed because of the place of
+ |hpack_filter| in the source of \LuaTeX. We will discuss more detail at Subsection~\ref{ssec-stack}.
+
+\item[Insertion of Glues/Kerns for Japanese Typesetting]
+This part is already described at Subsection~\ref{ssec-jglue}.
+
+\item[Adjustument of Places of (Japanese) Characters]
+Under \LuaTeX-ja, the size of the virtual body of a Japanese character
+ and its position (\emph{i.e.}, offset) are determined by the
+ metric, since the optimal width of a character in
+ typesetting---in most cases, this is specified width in the
+ metric---and the actual width in TrueType/Opentype fonts
+ often differ. For example, the width the fullwidth open brace
+ `\inhibitglue ｛' is considered to be half-width in
+ typesetting, although this character is full-width in
+ TrueType fonts like IPA~Mincho.
+
+To adjust size/places of Japanese characters, \LuaTeX-ja encapsules a
+ \textit{glyph\_node} which containing a Japanese character
+ into a horizontal box which size is specified in the metric.
+\end{description}
+
+\subsection{Stack Management}
+\label{ssec-stack}
+\LuaTeX-ja has a lot of parameters for Japanese typesetting.
+
+\subsection*{About the Project}
+\subsection*{Acknowledgements}
+
+
+%%% The style of the bibiliogrphy is `amsplain'.
+\providecommand{\bysame}{\leavevmode\hbox to3em{\hrulefill}\thinspace}
+\providecommand{\href}[2]{#2}
+\begin{thebibliography}{9}
+
+%\bibitem{Knuth}
+%Donald E.~Knuth, \emph{The \TeX book}, Addison-Wesley, 1986.
+
+\bibitem{ptex}
+ASCII MEDIA WORKS, \textbf{アスキー日本語\TeX\ (p\TeX)}\ (in Japanese). \url{http://ascii.asciimw.jp/pb/ptex/}
+
+%\bibitem{Eijkhout}
+%Victor Eijkhout, \emph{\TeX\ by Topic, A \TeX nician's Reference}, Addison-Wesley, 1992. \url{http://www.cs.utk.edu/~eijkhout/texbytopic-a4.pdf}
+
+\bibitem{luaums}
+Hironori Kitagawa, \textbf{LuaTeXで日本語}\ (in Japanese). \url{http://oku.edu.mie-u.ac.jp/tex/mod/forum/discuss.php?d=378}
+
+\bibitem{luajalayout}
+Kazuki Maeda\ (前田一貴), \textbf{luajalayout パッケージ —LuaLaTeX による日本語組版—}\ (in Japanese).
+\url{http://www-is.amp.i.kyoto-u.ac.jp/lab/kmaeda/lualatex/luajalayout/}
+
+\bibitem{luajp-test}
+Atsuhito Kohda, \textbf{LuaTeXと日本語}\ (in Japanese). \url{http://www1.pm.tokushima-u.ac.jp/~kohda/tex/luatex-old.html}
+
+\bibitem{joylua}
+Yannis Haralambous. \textbf{The Joy of LuaTeX}. \url{http://luatex.bluwiki.com/}
+
+\bibitem{luatexref}
+\textbf{The \LuaTeX reference}
+
+\end{thebibliography}
+
+\end{document}
OSDN Git Service
