1 %#!lualatex ajt-devel-ltja
\r
4 %%% Packages used in this paper
\r
6 %%% Font setting for \LuaTeX; this is extract from ajt.cls
\r
9 \RequirePackage{fontspec,xunicode}
\r
10 \RequirePackage{luatextra}
\r
11 \setmainfont[Mapping=tex-text]{Palatino LT Std}
\r
12 \setsansfont[Mapping=tex-text]{Optima LT Std}
\r
14 \RequirePackage{fontspec,luatextra}
\r
15 \setmainfont[Mapping=tex-text]{TeX Gyre Pagella} % \simeq Palatino
\r
19 \usepackage{luatexja,luatexja-fontspec}
\r
20 \ltjsetparameter{jacharrange={-3,-8}}
\r
21 \DeclareFontShape{JY3}{mc}{m}{n}{<-> s*[0.92489] file:ipam.ttf:jfm=ujis}{}
\r
22 \DeclareFontShape{JY3}{gt}{m}{n}{<-> s*[0.92489] file:ipag.ttf:jfm=ujis}{}
\r
23 % quick hack: monospaced Japanese font by \ttfamily
\r
24 \DeclareKanjiFamily{JY3}{\ttdefault}{}{}
\r
25 \DeclareFontShape{JY3}{\ttdefault}{m}{n}{<-> s*[0.92489] file:ipag.ttf:jfm=mono}{}
\r
28 %%% LTXexample environment
\r
29 \usepackage{showexpl,lltjlisting}
\r
30 \lstset{basicstyle=\ttfamily\small, width=0.3\textwidth, basewidth=.5em}
\r
32 %%% Verbatim environment
\r
33 \usepackage{fancyvrb}
\r
34 \CustomVerbatimEnvironment{code}{Verbatim}%
\r
35 {numbers=left,xleftmargin=1.5em,baselinestretch=1.069,fontsize=\small}
\r
36 \CustomVerbatimEnvironment{codewithoutnum}{Verbatim}%
\r
37 {xleftmargin=1.5em,baselinestretch=1.069,fontsize=\small}
\r
38 \CustomVerbatimEnvironment{codewithoutnumsmall}{Verbatim}%
\r
39 {xleftmargin=1.5em,baselinestretch=1.0,fontsize=\footnotesize}
\r
40 \DefineShortVerb{\|}
\r
43 \usepackage{mflogo,booktabs}
\r
44 \definecolor{grayx}{gray}{0.85}
\r
46 %%% Mandatory article metadata %%%
\r
47 \title{Development of \LuaTeX-ja package}
\r
48 \author{Hironori Kitagawa {\normalsize 北川 弘典}}
\r
49 \address{\LuaTeX-ja project team}
\r
50 \email{h\_kitagawa2001@yahoo.co.jp}
\r
52 \keywords{\TeX, p\TeX, \LuaTeX, \LuaTeX-ja, Japanese}
\r
54 \LuaTeX-ja package is a macro package for typesetting Japanese
\r
55 documents under \LuaTeX. The package has more flexibility of
\r
56 typesetting than \pTeX, which is widely used Japanese extension of \TeX,
\r
57 and has corrected some unwanted features of \pTeX.
\r
58 In this paper, we describe specifications, the current status and some
\r
59 internal processing methods of \LuaTeX-ja.
\r
62 \newcommand{\parname}[1]{\textsf{#1}}
\r
63 \newcommand{\jstrut}{\vrule width0pt height\cht depth\cdp}
\r
64 \newcommand{\imagfm}[1]{\ifvmode\leavevmode\fi%
\r
65 \hbox{\fboxsep=0pt\fbox{\setbox0=\hbox{#1}\copy0\kern-\wd0
\r
66 \smash{\vrule width \wd0 height 0.4pt depth0.4pt}}}}
\r
69 %%% Do not forget to start with \maketitle!
\r
72 \section{Introduction}
\r
73 \subsection{History}
\r
74 To typeset Japanese documents with \TeX, ASCII \pTeX~\cite{ptex} has
\r
75 been widely used in Japan. There are other methods---for example, using
\r
76 Omega and OTP~\cite{omega}, or with the CJK package---to do so, however,
\r
77 these alternative methods did not become a majority. The author thinks
\r
78 that this is because \pTeX\ enables us to produce high-quality documents
\r
79 (e.g.,~supporting vertical typesetting), and the appearance of \pTeX\ is
\r
80 earlier than that of alternatives described above.
\r
82 However, \pTeX\ has been left behind from the extensions of \TeX\
\r
83 such as \eTeX\ and \pdfTeX, and the diffusion of UTF-8 encoding. In
\r
84 recent years, the situation has become better, because of development
\r
85 of |ptexenc|~\cite{ptexenc} by Nobuyuki Tsuchimura (\hbox{土村展之}),
\r
86 $\varepsilon$-\pTeX~\cite{eptex} by the author,~and u\pTeX~\cite{uptex}
\r
87 by Takuji Tanaka (田中琢爾). However, continuing this approach, namely, to develop
\r
88 an engine extension localized for Japanese, is not wise. This approach
\r
89 needs lots of work for \emph{each} engine, and since \LuaTeX\ has an ability
\r
90 to hook \TeX's internal process by using Lua callbacks, the necessity of
\r
91 an engine extension is getting smaller.
\r
94 There were several experimental attempts to typeset
\r
95 Japanese documents with \LuaTeX\ before. Here we cite three examples:
\r
97 \item |luaums.sty|~\cite{luaums} developed by the author. This
\r
98 experimental package is for creating a certain Japanese-based presentation
\r
100 \item the \emph{luajalayout} package~\cite{luajalayout}, formerly known as the
\r
101 \emph{jafontspec} package, by Kazuki Maeda (前田一貴). This package is based on
\r
102 \LaTeXe\ and \emph{fontspec} package.
\r
103 \item the \emph{luajp-test} package~\cite{luajp-test}, a test package made by
\r
104 Atsuhito Kohda (香田温人), based on articles on the web page~\cite{joylua}.
\r
106 However, these packages are based on \LaTeXe, and do not have much
\r
107 ability to control the typesetting rule. And it is inefficient that more
\r
108 than one people separately develop similar packages. Development of the
\r
109 \LuaTeX-ja package is started initially by the author and Kazuki Maeda, because of
\r
112 \subsection{Development policy of \LuaTeX-ja}
\r
114 The first aim of \LuaTeX-ja project is to implement features (from the
\r
115 `primitive' level) of \pTeX\ as macros under \LuaTeX, so \LuaTeX-ja is
\r
116 much affected by \pTeX. However, as development proceeds, some
\r
117 technical/conceptual difficulties are arisen. Hence we changed the aim
\r
118 of the project as follows:
\r
120 \item\emph{\LuaTeX-ja offers at least the same flexibility of
\r
121 typesetting that p\TeX\ has.}
\r
123 We think that the ability of producing outputs conformed to
\r
124 JIS~X~4051~\cite{jisx4051}, the Japanese Industrial Standard for
\r
125 typesetting, or to a technical note~\cite{w3c} by W3C is not enough;
\r
126 if one wants to produce very incoherent outputs for some reason, it
\r
127 should be possible.
\r
128 In this point, previous attempts of Japanese typesetting with \LuaTeX\
\r
129 which we cited in the previous subsection are inadequate.
\r
131 \pTeX\ has some flexibility of typesetting, by changing internal
\r
132 parameters such as |\kanjiskip| or |\prebreakpenalty|, and by using
\r
133 custom JFM (Japanese TFM). Therefore we decided to include these
\r
134 functionality to \LuaTeX-ja.
\r
136 \item\emph{\LuaTeX-ja isn't mere re-implementation or porting of \pTeX;
\r
137 some (technically and/or conceptually) inconvenient features of
\r
138 \pTeX\ are modified.}
\r
140 We describe this point in more detail at the next section.
\r
144 \subsection{Overview of the processes}
\r
146 We describe an outline of \LuaTeX-ja's process in order.
\r
148 \item In the |process_input_buffer| callback: treatment of breaking
\r
149 lines after a Japanese character (in Subsection~\ref{ssec-line}).
\r
151 \item In the |hyphenate| callback: font replacement.
\r
153 \LuaTeX-ja looks into for each \textit{glyph\_node}~$p$ in the list. If
\r
154 the character represented by $p$ is considered as a Japanese
\r
155 character, the font used in $p$ is replaced by the value of
\r
156 |\ltj@curjfnt|, an attribute for `the current Japanese font'
\r
159 Furthermore the subtype of $p$ is subtracted by 1 to suppress
\r
160 hyphenation around it by \LuaTeX, because later processes of
\r
161 \LuaTeX-ja take care of all things about Japanese characters.
\r
163 \item In |pre_linebreak_filter| and |hpack_filter| callbacks:
\r
166 \item \LuaTeX-ja has its own stack system, and the current horizontal
\r
167 list is traversed in this stage to determine what is the level of
\r
168 \LuaTeX-ja's internal stack at the end of the list (in
\r
169 Subsection~\ref{ssec-stack}).
\r
171 \item In this stage, \LuaTeX-ja inserts glues/kerns for Japanese
\r
172 typesetting in the list. This is the core of \LuaTeX-ja (in
\r
173 Subsection~\ref{ssec-jglue}).
\r
175 \item To make a match between a metric and a real font, sometimes
\r
176 adjustument of the position of (Japanese) glyphs are performed
\r
177 (Subsection~\ref{ssec-width}).
\r
179 \item In the |mlist_to_hlist| callback: replacement of Japanese characters in math formulas.
\r
180 This stage is similar to adjustument of the position of glyphs (see
\r
181 above), so we omit it from this paper.
\r
184 \subsection{Contents of this paper}
\r
185 Here we describe the contents of the rest of this paper briefly. In
\r
186 Section~\ref{sec:differences_with_ptex},
\r
187 we describe major differences between \pTeX\ and \LuaTeX-ja.
\r
188 The next section, Section~\ref{sec:distinction_of_characters},
\r
189 is concentrated on a problem `how we
\r
190 distinguish between Japanese characters and alphabetic characters'. In
\r
191 Section~\ref{sec:current_status}, we show rest of features of \LuaTeX-ja package, and
\r
192 current status of the package. Finally, in Section~\ref{sec:implementation}, we describe some
\r
193 internal routines of \LuaTeX-ja.
\r
195 \subsection{General information of the project}
\r
196 This \LuaTeX-ja project is hosted by SourceForge.jp. The official wiki
\r
198 \url{http://sourceforge.jp/projects/luatex-ja/wiki/}. There is
\r
199 no stable version on October 15, 2011, however a set of developer sources can be
\r
200 obtained from the git repository. Members of the project team are as follows
\r
201 (in random order): Hironori Kitagawa, Kazuki Maeda, Takayuki Yato,
\r
202 Yusuke Kuroki, Noriyuki Abe, Munehiro Yamamoto, Tomoaki Honda,
\r
203 and~Shuzaburo Saito.
\r
206 \section{Major differences with \pTeX}
\r
207 \label{sec:differences_with_ptex}
\r
208 In this section, we explain several major differences between \pTeX\
\r
209 and our \LuaTeX-ja. For general information of Japanese typesetting and the
\r
210 overview of \pTeX, please see Okumura~\cite{ptexjp}.
\r
213 \subsection{Names of control sequences}
\r
214 \label{ssec-csname} Because \pTeX\ is an engine modification of Knuth's
\r
215 original \TeX82 engine, some primitives added by it take a form that is
\r
216 very difficult to be simulated by a macro. For example, an additional
\r
217 primitive |\prebreakpenalty|$\langle\hbox{\it
\r
218 char\_code}\rangle$|[=]|$\langle\hbox{\it penalty}\rangle$ in \pTeX\
\r
219 sets the amount of penalty inserted before a character whose code is
\r
220 $\langle\hbox{\it char\_code}\rangle$ to $\langle\hbox{\it
\r
221 penalty}\rangle$, and this form |\prebreakpenalty|$\langle\hbox{\it
\r
222 char\_code}\rangle$ can be also used for retrieving the value.
\r
224 Moreover, there are some parameters which values of them at the end of a
\r
225 horizontal box or that of a paragraph are effective in whole box or
\r
226 paragraph. These parameters were implemented as additional internal
\r
227 parameters in \pTeX. However, the implementation of these parameters in
\r
228 \LuaTeX-ja is not so easy; we will discuss it in
\r
229 Subsection~\ref{ssec-stack}.
\r
231 From above two~problems we discussed above, the assignment and retrieval
\r
232 of most parameters in \LuaTeX-ja are summarized into the following
\r
233 three~control sequences:
\r
235 \item |\ltjsetparameter{|$\langle\hbox{\it
\r
236 name}\rangle$|=|$\langle\hbox{\it value}\rangle$|,...}|: for local
\r
238 \item |\ltjglobalsetparameter|: for global assignment. These two control
\r
239 sequences obey the value of |\globaldefs| primitive.
\r
240 \item |\ltjgetparameter{|$\langle\hbox{\it
\r
241 name}\rangle$|}[{|$\langle\hbox{\it optional
\r
242 argument}\rangle$|}]|: for retrieval. The returned value is always
\r
246 \subsection{Line-break after a Japanese character}
\r
249 Japanese texts can break lines almost everywhere, in contrast with
\r
250 alphabetic texts can break lines only between words (or use
\r
251 hyphenation). Hence, \pTeX's input processor is modified so that a
\r
252 line-break after a Japanese character doesn't emit a space. However,
\r
253 there is no way to customize the input processor of \LuaTeX, other than
\r
254 to hack its CWEB-source. All a macro package can do is to modify an input line before
\r
255 when \LuaTeX\ begin to process it, inside the |process_input_buffer|
\r
258 Hence, in \LuaTeX-ja, a comment letter (we reserve U+FFFFF for this
\r
259 purpose) will be appended to an input line, if this line ends with a Japanese
\r
260 character.\footnote{Strictly speaking, it also requires that the catcode
\r
261 of the end-line character is 5~(\emph{end-of-line}). This condition is
\r
262 useful under the verbatim environment.} One might jump to a conclusion
\r
263 that the treatment of a line-break by \pTeX\ and that of \LuaTeX-ja are
\r
264 totally same, however they are different in the respect that \LuaTeX-ja's
\r
265 judgement whether a comment letter will be appended the line is done
\r
266 \emph{before} the line is actually processed by \LuaTeX.
\r
268 Figure~\ref{fig-linebreak} shows an example of this situation; the
\r
269 command at the first line marks most of Japanese characters as
\r
270 `non-Japanese characters'. In other words, from that command onward, the
\r
271 letter `あ' will be treated as an alphabetic character by
\r
272 \LuaTeX-ja. Then, it is natural to have a space between `あ' and `y' in
\r
273 the output, where the actual output in the figure does not so. This is
\r
274 because `あ' is considered a Japanese character by \LuaTeX-ja,
\r
275 when \LuaTeX-ja does a decision whether U+FFFFF will be added to the
\r
280 \font\x=IPAMincho \x
\r
281 \ltjsetparameter{jacharrange={-6}}xあ
\r
284 \caption{A notable sample showing the treatment of a line-break after a
\r
285 Japanese character.}\label{fig-linebreak}
\r
288 \subsection{Separation between `real' fonts and metrics}
\r
289 \label{ssec-sepmet}
\r
291 Traditionally, most Japanese fonts used in typesetting are not
\r
292 proportional, that is, most glyphs have same size (in most cases,
\r
293 square-shaped). Hence, it is not rare that the contents of different
\r
294 JFMs are essentially same, and only differ in their names. For example,
\r
295 |min10.tfm| and |goth10.tfm|, which are JFMs shipped with \pTeX\ for
\r
296 seriffed \emph{mincho} family and sans-seriffed \emph{gothic} family,
\r
297 differ their |FAMILY| and |FACE| only. Moreover, |jis.tfm| and
\r
298 |jisg.tfm|, which consists a parts of \emph{jis} font metric, which is
\r
299 used in \emph{jsclasses}~\cite{jsclasses} by Haruhiko Okumura (奥村晴彦),
\r
300 are totally same as binary files. Considering this situation, we
\r
301 decided to separate `real' fonts and metrics used for them in
\r
302 \LuaTeX-ja. Typical declarations of Japanese fonts in the style of plain
\r
303 \TeX\ are shown in Figure~\ref{fig-jfdef}. We would like to add several
\r
306 \item A control sequence |\jfont| must be used for Japanese fonts, instead of |\font|.
\r
307 \item \LuaTeX-ja automatically loads the \emph{luaotfload} package, so
\r
308 |file:| and |name:| prefixes, and various font features can be
\r
309 used as the line~1 in Figure~\ref{fig-jfdef}.
\r
310 \item The |jfm| key specifies the metric for the font. In
\r
311 Figure~\ref{fig-jfdef}, both fonts will use a metric stored in a
\r
312 Lua script named |jfm-ujis.lua|. This metric is the standard
\r
313 metric in \LuaTeX-ja, and is based on JFMs used in the \emph{otf}
\r
314 package~\cite{otf}.
\r
315 \item The |psft:| prefix can be used to specify name-only, non-embedded
\r
316 fonts. When one display a pdf with these fonts, actual fonts which
\r
317 will be used for them depend on a pdf reader.
\r
319 The specification of a metric for \LuaTeX-ja is similar to that of a JFM
\r
320 (see \cite{ptexjp}); characters are grouped into several classes, the
\r
321 size information of characters are specified for each class, and
\r
322 glue/kern insertions are specified for each pair of classes. Although
\r
323 the author have not tried, it may be possible to develop a program that
\r
324 `converts' a JFM to a metric for \LuaTeX-ja. \LuaTeX-ja offers three
\r
325 metrics by default; |jfm-ujis.lua|, |jfm-jis.lua| based on the
\r
326 \emph{jis} font metric, and |jfm-min.lua| based on old |min10.tfm|.
\r
328 Note that |-kern| in features
\r
329 is important, because kerning information from real font itself will
\r
330 clash with glue/kern informations from the metric.
\r
334 \jfont\foo=file:ipam.ttf:jfm=ujis;script=latn;-kern;+jp04 at 12pt
\r
335 \jfont\bar=psft:Ryumin-Light:jfm=ujis at 10pt
\r
337 \caption{Typical declarations of Japanese fonts.}
\r
341 \subsection{Insertion of glues/kerns for Japanese typesetting: timing}
\r
344 As described in \cite{luatexref}, \LuaTeX's kerning and ligaturing
\r
345 processes are totally different from those of \TeX82. \TeX82's process is
\r
346 done just when a (sequence of) character is appended to the current
\r
347 list. Thus we can interrupt this process by writing as
\r
348 |f{}irm|. However, \LuaTeX's process is \emph{node-based}, that is, the
\r
349 process will be done when a horizontal box or a paragraph is ended, so
\r
350 |f{}irm| and |firm| yield same outputs under \LuaTeX.
\r
352 The situation for Japanese characters is more complicated.
\r
353 Glues (and kerns) which are needed for Japanese
\r
354 typesetting will be divided into the following three categories:
\r
356 \item Glue (or kern) from the metric of Japanese fonts (\emph{JFM glue},
\r
359 \item Default glue between a Japanese character and an alphabetic
\r
360 character (\emph{xkanjiskip}, for short), usually 1/4 of
\r
361 full-width (\emph{shibuaki}) with some stretch and shrink for
\r
362 justifying each line.
\r
363 \item Default glue between two consecutive Japanese characters
\r
364 (\emph{kanjiskip}, for short). The main reason of this glue is to
\r
365 enable breaking lines almost everywhere in Japanese texts. In most
\r
366 cases, its natural width is zero, and some stretch/shrink for
\r
367 justifying each line.
\r
369 In \pTeX, these three kinds of glues are treated differently. A JFM glue
\r
370 is inserted when a (sequence of) Japanese character is appended to the
\r
371 current list, same as the case of alphabetic characters in \TeX82. This
\r
372 means that one can interrupt the insertion process by saying |{}|. A
\r
373 \emph{xkanjiskip} is inserted just before `hpack' or line-breaking of a
\r
374 paragraph; this timing is somewhat similar to that of \LuaTeX's kerning
\r
375 process. Finally, A \emph{kanjiskip} is not appeared as a node anywhere;
\r
376 only appears implicitly in calculation of the width of a horizontal box,
\r
377 that of breaking lines, and the actual output process to a DVI
\r
378 file. These specifications made \pTeX's behavior very hard to
\r
381 \LuaTeX-ja inserts glues in all three categories simultaneously inside
\r
382 |hpack_filter| and |pre_linebreak_filter| callbacks. The reasons of
\r
383 this specification are to behave like alphabetic characters in \LuaTeX\
\r
384 (as described in the first paragraph), and to clarify the specification
\r
385 for \LuaTeX-ja's process.
\r
387 \subsection{Insertion of glues/kerns for Japanese typesetting: specification}
\r
389 \caption{Examples of differences between \pTeX\ and \LuaTeX-ja.}
\r
390 \label{tab-jfmglue}
\r
392 \begin{tabular}{llllllll}
\r
394 &\multicolumn{1}{c}{(1)}&\multicolumn{1}{c}{(2)}&\multicolumn{1}{c}{(3)}&\multicolumn{1}{c}{(4)}\\
\r
395 Input &|あ】{}【〙\/〘| &|い』\/a| &|う）\hbox{}（| &|え］\special{}［|\\\midrule
\r
396 \pTeX &あ】\hbox{}【〙\hbox{}〘&い』\/a &う）\hbox{}（ &え］\hbox{}［\\
\r
397 \LuaTeX-ja &あ】{}【〙\/〘 &い』\/a &う）\hbox{}（ &え］\special{}［\\
\r
405 \fontsize{40}{40}\selectfont
\r
406 \imagfm{\jstrut あ}%
\r
407 \imagfm{\jstrut 】\inhibitglue}%
\r
408 \imagfm{\jstrut\kern.5\zw}%
\r
409 \imagfm{\jstrut\kern.5\zw}%
\r
410 \imagfm{\jstrut\inhibitglue【}%
\r
411 \imagfm{\jstrut 〙\inhibitglue}%
\r
412 \imagfm{\jstrut\kern.5\zw}%
\r
413 \imagfm{\jstrut\kern.5\zw}%
\r
414 \imagfm{\jstrut\inhibitglue〘}%
\r
416 \caption{Detail of (1) in Table~\ref{tab-jfmglue}.}
\r
417 \label{fig-ptexjfm}
\r
420 Now we will take a look inside the insertion process itself, and describe 4~points.
\r
422 \begin{description}
\r
423 \item[Ignored Nodes]
\r
424 As noted in the previous subsection, the insertion process in \pTeX\ can
\r
425 be interrupted by saying |{}| or anything else\footnote{This
\r
426 is why some tricks like \texttt{ちょ\char`\{\char`\}っと} for
\r
427 \texttt{min10.tfm} and other `old' JFMs work.}. This leads
\r
428 the second row in Table~\ref{tab-jfmglue}, or
\r
429 Figure~\ref{fig-ptexjfm}. `The process is interrupted' means
\r
430 that \pTeX\ does not think the letter `】\inhibitglue' is
\r
431 followed by `\inhibitglue【', hence two half-width glues are
\r
432 inserted between between `】\inhibitglue' and `\inhibitglue【',
\r
433 where one is from `】\inhibitglue' and another is from
\r
436 On the other hand, in \LuaTeX-ja, the process is done inside
\r
437 |hpack_filter| and |pre_linebreak_filter| callbacks. Hence,
\r
438 \emph{anything that does not make any node will be
\r
439 ignored}\ in \LuaTeX-ja, as shown in (1) in
\r
440 Table~\ref{tab-jfmglue}. \LuaTeX-ja also ignores any nodes
\r
441 which does not make any contribution to current horizontal
\r
442 list---\emph{ins\_node}, \emph{adjust\_node},
\r
443 \emph{mark\_node}, \emph{whatsit\_node} and
\r
444 \emph{penalty\_node}---, as shown in (4).
\r
447 By the way, around a \emph{glyph\_node} $p$ there may be some nld odes
\r
448 attached to $p$. These are an accent and kerns for
\r
449 positioning it, and a kern from the italic
\r
450 correction\footnote{\TeX82 (and \LuaTeX) does not distinguish
\r
451 between explicit kern and a kern for italic correction. To
\r
452 distinguish them, an additional subtype for kern is introduced
\r
453 in \pTeX. On the other hand, \LuaTeX-ja uses an additional attribute and
\r
454 redefines \texttt{\char`\\/}.} for $p$. It is natural that
\r
455 these attachments should be ignored inside the process. Hence
\r
456 \LuaTeX-ja takes this approach, as the latest version of
\r
457 \pTeX\ (p3.2). This explains (2) in the figure.
\r
459 Summerizing above, one should put an empty horizontal box |\hbox{}| to
\r
460 where he wants to interrupt the insertion process in
\r
461 \LuaTeX-ja as (3) in the figure.
\r
463 \item[Fonts with the Same Metric]
\r
464 Recall that \LuaTeX-ja separated `real' fonts and metrics, as in Subsection~\ref{ssec-sepmet}.
\r
465 Consider the following input, where all Japanese fonts use same metric
\r
466 (in \LuaTeX-ja), and |\gt| selects \emph{gothic} family for
\r
467 the current Japanese font family:
\r
473 If the above input is processed by \pTeX, because the insertion process is
\r
474 interrupt by |\gt|, the result looks like
\r
476 \mc 明朝）\hbox{}\gt （ゴシック
\r
478 However this seems to be unnatural, since two Japanese fonts in the
\r
479 output use the same metric, i.e.,~the same
\r
480 typesetting rule. Hence, we decided that Japanese fonts with
\r
481 the same metric are treated as one font in the insertion
\r
482 process of \LuaTeX-ja. Thus, the output from the above input
\r
483 in \LuaTeX-ja looks like:
\r
487 One might have the situation that this default behavior is not
\r
488 suitable. \LuaTeX-ja offers a way to cope with this case, but
\r
489 we leave it to the manual~\cite{man}.
\r
491 \item[Fonts with Different Metrics]
\r
492 In the case where two consecutive Japanese characters use different metrics and/or
\r
493 different size is similar. Consider the following input where
\r
494 the \emph{mincho} family and the \emph{gothic} family use
\r
501 As the previous paragraph, this input yields the following, by \pTeX:
\r
503 \mc 漢）\hbox{}\gt （漢）\hbox{}\large （大
\r
505 We thought that amounts of spaces between parentheses in above output
\r
506 are too much. So we changed the default behavior of
\r
507 \LuaTeX-ja so that the amount of a glue between two Japanese
\r
508 characters with different metrics is the average of a glue
\r
509 from the left character and that from the right
\r
510 character. For example, Figure~\ref{fig-diffmet} shows the
\r
511 output from above input. The width of glue indicated `(1)' is
\r
512 $(a/2 + a/2)/2 = 0.5a$, and the width of glue indicated `(2)'
\r
513 is $(a/2 + 1.2a/2)/2 = 0.55a$. This default behavior can be
\r
514 changed by \textsf{diffrentmet} parameter of \LuaTeX-ja.
\r
518 \fontsize{40}{40}\selectfont
\r
519 \imagfm{\jstrut\smash{%
\r
520 \vtop{\lineskiplimit=\maxdimen\lineskip2pt\halign{#\cr漢\cr
\r
521 \small\vrule height .5ex depth .5ex\hrulefill\ \lower.5ex\hbox{$a$}\
\r
522 \hrulefill\vrule height .5ex depth .5ex\cr}}}}%
\r
523 \imagfm{\jstrut ）\inhibitglue}%
\r
524 \hbox to .5\zw{\hss\normalsize (1)\hss}%
\r
525 \imagfm{\jstrut\inhibitglue\gt （}%
\r
526 \imagfm{\jstrut\gt 漢}%
\r
527 \imagfm{\jstrut\gt ）\inhibitglue}%
\r
528 \hbox to .55\zw{\hss\normalsize (2)\hss}%
\r
529 \imagfm{\fontsize{48}{48}\selectfont\jstrut\gt\inhibitglue （}%
\r
530 \imagfm{\fontsize{48}{48}\selectfont\jstrut\smash{%
\r
531 \vtop{\lineskiplimit=\maxdimen\lineskip2pt\halign{#\cr\gt 大\cr
\r
532 \small\vrule height .5ex depth .5ex\hrulefill\ \lower.5ex\hbox{$1.2a$}\
\r
533 \hrulefill\vrule height .5ex depth .5ex\cr}}}}
\r
535 \caption{Fonts with different metrics.}
\r
536 \label{fig-diffmet}
\r
539 \item[\emph{kanjiskip} and \emph{xkanjiskip}]
\r
540 In \pTeX, the value of \emph{xkanjiskip} is controlled by a skip named
\r
541 |\xkanjiskip|. A defect of this implementation is that the
\r
542 value of \emph{xkanjiskip} is not connected with the size of
\r
543 the currnt Japanese font. It seems that |EXTRASPACE|,
\r
544 |EXTRASTRETCH|, |EXTRASHRINK| parameters in a JFM are
\r
545 reserved for specifying the default value of
\r
546 \emph{xkanjiskip} in a unit of the design size, but \pTeX\
\r
547 did not use these parameters.
\r
549 Considering this situation of p\TeX, \LuaTeX-ja can use the value of
\r
550 \emph{xkanjiskip} that specified in a metric. If the value of
\r
551 \emph{xkanjiskip} on user side (this is the
\r
552 \textsf{xkanjiskip} parameter in |\ltjsetparameter|) is
\r
553 |\maxdimen|, then \LuaTeX-ja use the specification from
\r
554 the current used metric as the actual value of
\r
556 This description also applies for \emph{kanjiskip}.
\r
559 \section{Distinction of characters}
\r
560 \label{sec:distinction_of_characters}
\r
561 Since \LuaTeX\ can handle Unicode characters natively, it is a major
\r
562 problem that how we distinguish Japanese characters and alphabetic
\r
563 characters. For example, the multiplication sign (U+00D7) exists both in
\r
564 ISO-8859-1 (hence in Latin-1 Supplement in Unicode) and in the basic
\r
565 Japanese character set JIS~X~0208. It is not desirable that this
\r
566 character is treated as an alphabetic char, because this symbol is often
\r
567 used in the sense of `negative' in Japan.
\r
569 \subsection{Character ranges}
\r
570 Before we describe the approach taken is \LuaTeX-ja, we review the
\r
571 approach taken by u\pTeX. u\pTeX\ extends the |\kcatcode| primitive in
\r
572 \pTeX, to use this primitive for setting how a character is treated
\r
573 among alphabetic characters~(15), \emph{kanji}~(16), \emph{kana}~(17),
\r
574 \emph{kanji}, \emph{Hangul}~(17), or~\emph{other CJK characters}~(18).
\r
575 The assignment to |\kcatcode| can be done by a Unicode
\r
576 block\footnote{There are some exceptions. For example, U+FF00--FFEF
\r
577 (Halfwidth and Fullwidth Forms) are divided into three blocks in recent
\r
580 \LuaTeX-ja adopted a different approach. There are many Unicode blocks
\r
581 in Basic Multilingual Plane which are not included in
\r
582 Japanese fonts, it is inconvenient if we treat by a Unicode
\r
583 block. Furthermore, JIS~X~0208 are not just union of Unicode
\r
584 blocks; for example, the intersection of JIS~X~0208 and
\r
585 Latin-1 Supplement is shown in
\r
586 Table~\ref{tab-inter}. Considering these two points, to
\r
587 customize the range of Japanese characters in \LuaTeX-ja, one
\r
588 has to define ranges of character codes in his source in advance.
\r
592 \caption{Intersection of JIS~X~0208 and Latin-1 Supplement.}
\r
595 \begin{tabular}{llll}
\r
596 \ltjjachar"A7 (U+00A7),&
\r
597 \ltjjachar"A8 (U+00A8),&
\r
598 \ltjjachar"B0 (U+00B0),&
\r
599 \ltjjachar"B1 (U+00B1),\\
\r
600 \ltjjachar"B4 (U+00B4),&
\r
601 \ltjjachar"B6 (U+00B6),&
\r
602 \ltjjachar"D7 (U+00D7),&
\r
603 \ltjjachar"F7 (U+00F7)
\r
610 We note that \LuaTeX-ja offers two additional control sequence,
\r
611 |\ltjjachar| and |\ltjalchar|. They are similar to |\char|
\r
612 primitive, but |\ltjjachar| always yields a Japanese character (if
\r
613 the argument is more than or equal to 128) and |\ltjalchar| always
\r
614 yields an alphabetic character, regardless of the argument.
\r
616 \subsection{Default setting of ranges}
\r
617 Patches for plain \TeX\ and \LaTeXe of \LuaTeX-ja predefines 8~character
\r
618 ranges, as shown in Table~\ref{tab-chrrng}. Almost of these ranges are
\r
619 just the union of Unicode blocks, and determined from the Adobe-Japan1-6
\r
620 character collection~\cite{aj16}, and JIS~X~0208. Among these 8~ranges,
\r
621 the ranges~2, 3, 6, 7, and~8 are considered ranges of Japanese
\r
622 characters, and others are considered ranges of alphabetic
\r
623 characters\footnote{Note that ranges 3~and~8 are considered ranges of
\r
624 alphabetic characters in this paper.}. We remark on ranges 2~and~8:
\r
625 \begin{description}
\r
627 JIS~X~0208 includes Greek letters and Cyrillic letters, however, these
\r
628 letters cannot be used for typesetting Greek or Russian, of
\r
629 course. Hence it is reasonable that Greek letters and
\r
630 Cyrillic consist another character range.
\r
631 \item[The range~8]
\r
632 If one want to use 8-bit TFMs, such as T1 or TS1 encodings, he should
\r
633 mark this range~8 as a range of alphabetic characters by
\r
635 |\ltjsetparameter{jacharrange={-8}}|
\r
637 This is because some 8-bit TFMs have a glyph in this range; for example,
\r
638 the character `\OE' is located at |"D7| in the T1 encoding. %"
\r
643 \caption{Predefined ranges in \LuaTeX-ja.}
\r
646 \begin{tabular}{@{\bf}rl}
\r
647 1&(Additional) Latin characters which are not belonged in the range~8.\\
\r
648 2&Greek and Cyrillic letters.\\
\r
649 3&Punctuations and miscellaneous symbols.\\
\r
650 4&Unicode blocks which does not intersect with Adobe-Japan1-6.\\
\r
651 5&Surrogates and supplementary private use Areas.\\
\r
652 6&Characters used in Japanese typesetting.\\
\r
653 7&Characters possibly used in CJK typesetting, but not in Japanese.\\
\r
654 8&Characters in Table~\ref{tab-inter}.
\r
659 \subsection{Control sequences producing Unicode characters}
\r
660 \label{ssec-unichar}
\r
662 The \emph{fontspec} package\footnote{Preciously
\r
663 saying, it is the \emph{xunicode} package, originally a package for
\r
664 \XeTeX and automatically loaded by the \emph{fontspec} package.} offer
\r
665 various control sequences that produce Unicode characters. However, they as
\r
666 it stands cannot work with the default range setting of \LuaTeX-ja. For
\r
667 example, |\textquotedblleft| is just an abbreviation of
\r
668 |\char"201C\relax| %"
\r
669 and the character U+201C (LEFT DOUBLE QUOTATION
\r
670 MARK) is treated as an Japanese character, because it belongs to the
\r
672 This problem is resolved by using |\ltjalchar| instead of the |\char| primitive.
\r
673 It is included in an optional package named \texttt{luatexja-\penalty0fontspec.sty}.
\r
674 Figure~\ref{fig-unitxt} ...
\r
678 ×, \char`×, % depend on range setting
\r
679 \ltjalchar`×, % alphabetic char
\r
680 \ltjjachar`×, % Japanese char
\r
681 \texttimes % alph. char (by fontspec)
\r
683 \caption{Control sequences producing a Unicode character.}
\r
687 The situation looks similar in math formulas, but in fact it differs.
\r
688 Control sequences that represents ordinary symbols defined by the
\r
689 \emph{unicode-math} package is just synonym of a character. For example,
\r
690 the meaning of |\otimes| is just the character U+2297 (CIRCLED TIMES),
\r
691 which is included in the range~3. However, it is difficult to define a
\r
692 control sequence like |\ltjalUmathchar| as a counterpart of
\r
693 |\Umathchar|, since an input like `|\sum^\ltjalUmathchar ...|' has to be
\r
696 However, we couldn't include a solution to this problem in time for this
\r
697 paper, due to a lack of time. We are just testing a solution that we
\r
698 will explain it below:
\r
700 \item \LuaTeX-ja has a list of character codes which will be treated as
\r
701 alphabetic characters in math mode. Considering 8-bit TFMs for
\r
702 math symbols, this list includes natural numbers between |"80| and
\r
704 \item Redefine internal commands defined in the \emph{unicode-math}
\r
706 codes of characters which are mentioned in the \emph{unicode-math}
\r
707 package will be included in the list.
\r
711 We would like to extend treatments described in this section to 8-bit
\r
712 font encodings, but we leave it to further development too.
\r
714 \section{Current status of development}
\r
715 \label{sec:current_status}
\r
716 At the moment, \LuaTeX-ja can be used under plain \TeX, and under
\r
717 \LaTeXe. Generally speaking, one only has to read |luatexja.sty|, by
\r
718 |\input| command or |\usepackage| (in~\LaTeXe), if you merely want to
\r
719 typeset Japanese characters. We look more detail by parts.
\r
721 \subsection{`Engine extension'}
\r
722 The lowest part of \LuaTeX-ja corresponds the \pTeX\ extension as
\r
723 \emph{an engine extension of \TeX}. We, the project menbers, think that
\r
724 this part is almost done. There is one more feature of \LuaTeX-ja which
\r
725 we are going to explain:
\r
727 \begin{description}
\r
728 \item[Shifting Baseline]
\r
729 In order to make a match between Japanese fonts and alphabetic fonts,
\r
730 sometimes shifting the baseline of alphabetic characters may
\r
731 be needed. \pTeX\ has a dimension |\ybaselineshift|, which
\r
732 corresponds the amount of shifting down the baseline of alphabetic
\r
733 characters. This is useful for Japanese-based documents, but
\r
734 not for documents mainly in languages with alphabetic
\r
737 Hence, \LuaTeX-ja extends \pTeX's |\ybaselineshift| to Japanese
\r
738 characters. Namely, \LuaTeX-ja offers two parameters,
\r
739 \textsf{yjabaselineshift} and \textsf{yalbaselineshift}, for the
\r
740 amount of shifting the baseline of Japanese characters and
\r
741 that of alphabetic characters, respectively.
\r
744 \fontsize{40}{40}\selectfont\fboxsep0mm
\r
745 \vrule width 0.9\textwidth height0.4pt depth0.4pt\kern-0.9\textwidth
\r
746 \hbox to 0.9\linewidth{%
\r
748 \raise-10pt\imagfm{\jstrut 漢}%
\r
749 \raise-10pt\imagfm{\jstrut 字}\hskip.25\zw%
\r
753 \imagfm{\jstrut 漢}%
\r
754 \imagfm{\jstrut 字}\hskip.25\zw%
\r
755 \raise-10pt\imagfm{p}%
\r
756 \raise-10pt\imagfm{h}%
\r
761 \caption{First example of shifting baseline.}
\r
767 \fontsize{30}{30}\selectfont\fboxsep0mm
\r
768 \vrule width 0.9\textwidth height0.4pt depth0.4pt\kern-0.9\textwidth
\r
769 \hbox to 0.9\linewidth{%
\r
772 \imagfm{b}\hskip.25\zw%
\r
773 \imagfm{\jstrut 本}%
\r
774 \imagfm{\jstrut 文}\hskip.33333\zw%
\r
775 \raise3.514582pt\imagfm{\fontsize{20}{20}\selectfont\jstrut\inhibitglue （}%
\r
776 \raise3.514582pt\imagfm{\fontsize{20}{20}\selectfont\jstrut 注}%
\r
777 \raise3.514582pt\imagfm{\fontsize{20}{20}\selectfont\jstrut 釈}\hskip.1666667\zw%
\r
778 \raise3.514582pt\imagfm{\fontsize{20}{20}\selectfont c}%
\r
779 \raise3.514582pt\imagfm{\fontsize{20}{20}\selectfont o}%
\r
780 \raise3.514582pt\imagfm{\fontsize{20}{20}\selectfont m}%
\r
781 \raise3.514582pt\imagfm{\fontsize{20}{20}\selectfont m}%
\r
782 \raise3.514582pt\imagfm{\fontsize{20}{20}\selectfont e}%
\r
783 \raise3.514582pt\imagfm{\fontsize{20}{20}\selectfont n}%
\r
784 \raise3.514582pt\imagfm{\fontsize{20}{20}\selectfont t}%
\r
785 \raise3.514582pt\imagfm{\fontsize{20}{20}\selectfont\jstrut ）\inhibitglue}%
\r
787 \imagfm{\jstrut 本}%
\r
788 \imagfm{\jstrut 文}%
\r
793 \caption{Second example of shifting baseline.}
\r
797 An example output is shown in Figure~\ref{fig-bls}. The left half is the
\r
798 output when \textsf{yjabaselineshift} is positive, hence the
\r
799 baseline of Japanese characters is shifted down. On the other
\r
800 hand, the right half is the output when
\r
801 \textsf{yalbaselineshift} is positive, hence the baseline of
\r
802 alphabetic characters is shifted. Figure~\ref{fig-small}
\r
803 shows an intresting use of these parameters.
\r
806 Note that \LuaTeX-ja doesn't support vertical typesetting, \emph{tategaki}, for now.
\r
808 \subsection{Patches for plain \TeX\ and \LaTeXe}
\r
809 \pTeX\ has a patch for plain \TeX, namely |ptex.tex|, that for \LaTeXe\
\r
810 macro (this patch and \LaTeXe\ consist \emph{p\LaTeXe}), and
\r
811 |kinsoku.tex| which includes the default setting of \emph{kinsoku
\r
812 shori}, the Japanese hyphenation. We ported them to \LuaTeX-ja, except
\r
813 the codes related to vertical typesetting, because \LuaTeX-ja doesn't
\r
814 support vertical typesetting yet. We remark one point related to the
\r
816 \begin{description}
\r
818 \item[Behavior of\/ {\tt\char92fontfamily\/}]
\r
819 The control sequence |\fontfamily| in p\LaTeXe\ changes the current alphabetic
\r
820 font family and/or the current Japanese font family,
\r
821 depending the argument. More concretely,
\r
822 |\fontfamily{|$\langle\hbox{\it arg\/}\rangle$|}| changes the
\r
823 current alphabetic font family to $\langle\hbox{\it
\r
824 arg\/}\rangle$, if and only if one of the following
\r
825 conditions are satisfied:
\r
827 \item An alphabetic font family named $\langle\hbox{\it arg\/}\rangle$ in
\r
828 \emph{some} alphabetic encoding already defined in the document.
\r
829 \item There exists an alphabetic encoding $\langle\hbox{\it
\r
830 enc\/}\rangle$ already defined in the document such that a font
\r
831 definition file $\langle\hbox{\it enc\/}\rangle\langle\hbox{\it
\r
832 arg\/}\rangle$|.fd| (all lowercase) exists.
\r
834 The same criterion is used for changing Japanese font family.
\r
836 To work this behavior well, a list of all (alphabetic) encodings defined
\r
837 already in the document is needed. However, since \LuaTeX-ja
\r
838 is loaded as a package, \LuaTeX-ja cannot have this list.
\r
839 Hence \LuaTeX-ja adopted a different approach, namely
\r
840 |\fontfamily{|$\langle\hbox{\it arg\/}\rangle$|}| changes the
\r
841 current alphabetic font family to $\langle\hbox{\it
\r
842 arg\/}\rangle$, if and only if:
\r
844 \item An alphabetic font family named $\langle\hbox{\it arg\/}\rangle$
\r
845 in the current alphabetic encoding $\langle\hbox{\it
\r
846 enc\/}\rangle$ already defined in the document.
\r
847 \item A font definition file $\langle\hbox{\it enc\/}\rangle\langle\hbox{\it
\r
848 arg\/}\rangle$|.fd| (all lowercase) exists.
\r
856 \subsection{Classes for Japanese documents}
\r
857 To produce `high-quality' Japanese documents, we need not only that
\r
858 Japanese characters are correctly placed, but also class files for
\r
859 Japanese documents. In \pTeX, there are two major families of classes:
\r
860 \emph{jclasses} which is distributed with the official p\LaTeXe\ macros,
\r
861 and \emph{jsclasses}. At the present, \LuaTeX-ja
\r
862 simply contains their counterparts: \emph{ltjclasses} and
\r
863 \emph{ltjsclasses}. However, the policy on classess is not determined
\r
864 now, and we hope to have another family of classes which are useful in
\r
865 commercial printing. In the author's opinion, \emph{ltjclasses} is
\r
866 better to stay as an example of porting of class files for \pTeX\ to
\r
869 \subsection{Patches for packages}
\r
870 Apart from patches for the \LaTeXe~kernel and classes for Japanese
\r
871 documents, we need to make patches for several packages. At the present,
\r
872 we considered the following packages, and made patches or porting for
\r
873 the former two packages.
\r
875 \begin{description}
\r
876 \item[The \emph{fontspec} package] The \emph{fontspec} package is built
\r
877 on NFSS2, hence control sequences offered by the
\r
878 \emph{fontspec} package, such as |\setmainfont|, are only
\r
879 effective for alphabetic fonts if \LuaTeX-ja is loaded.
\r
880 \texttt{luatexja-\penalty0fontspec.sty} (not automatically
\r
881 loaded) offers these counterparts for Japanese fonts, with
\r
882 additional `j' in the name of control sequences, such as
\r
883 |\setmainjfont|. As described in
\r
884 Subsection~\ref{ssec-unichar}, it also includes a patch for
\r
885 control sequences producing Unicode characters.
\r
887 \item[The \emph{otf} package]
\r
888 This package is widely used in \pTeX\ for characters which is
\r
889 not in JIS~X~0208, and for using more than one weight in \emph{mincho}
\r
890 and \emph{gothic} font families. Therefore \LuaTeX-ja supports features
\r
891 in the \emph{otf} package, by loading \texttt{luatexja-\penalty0otf.sty}
\r
892 manually. Note that characters by |\UTF{xxxx}| and
\r
893 |\CID{xxxx}| are not appended to the current list as a
\r
894 \emph{glyph\_node}, so they are not affected by callbacks by
\r
895 the \emph{luaotfload} package. We have another remark; |\CID|
\r
896 does not work with TrueType fonts.
\r
898 \item[The \emph{listings} package]
\r
899 It is well-known that there is a patch |jlisting.sty| of the
\r
900 \emph{listings} package for p\LaTeXe. Generally speaking, it
\r
901 also can be used in \LuaTeX-ja. However, it seems to be that
\r
902 a Japanese character after a space does not recieve any
\r
903 process of the \emph{listings} package; this is inconvinient
\r
904 when we use the \emph{showexpl} package.
\r
909 \section{Implementation}
\r
910 \subsection{Handling of Japanese fonts}
\r
911 In \pTeX, there are three slots for maintaining current fonts, namely
\r
912 |\font| for alphabetic fonts, |\jfont| for Japanese font (in horizontal
\r
913 direction) and |\tfont| for Japanese font (in vertical direction). With
\r
914 these slots, we can manage the current font for alphabetic characters
\r
915 and that for Japanese characters separately in \pTeX. However, \LuaTeX\
\r
916 has only one slot for maintaining the current font, as \TeX82. This
\r
917 situation leads a problem: how can we maintain the `current Japanese
\r
920 There are three approaches for this problem. One approach is to make a
\r
921 mapping table from alphabetic fonts to corresponding Japanese fonts
\r
922 (here we don't assume that NFSS2 is available). Another approach is
\r
923 that we always use composite fonts with alphabetic fonts and Japanese
\r
924 fonts. The third approach is that the information of the current
\r
925 Japanese font is stored in an attribute. We adopted the third approach,
\r
926 since \LuaTeX-ja is much affected by \pTeX\ as we noted in
\r
927 Subsection~\ref{ssec-pol}.
\r
929 As in Figure~\ref{fig-jfdef}, \LuaTeX-ja uses |\jfont| for defining
\r
930 Japanese font, as \pTeX. However, because the information of the current
\r
931 Japanese font is stored into an attribute, control sequences defined by
\r
932 |\jfont| (e.g.,~|\foo| and |\bar| in Figure~\ref{fig-jfdef}) is
\r
933 not representing a font by the means of \TeX82. In other words, each of
\r
934 these control sequences is just an assignment to an attribute, therefore
\r
935 they cannot be an argument of |\the|, |\fontname|, nor |\textfont|.
\r
938 Callbacks by the \emph{luaotfload} package, e.g.,~replacement of glyphs
\r
939 according to font features, are executed just after `Examination of
\r
940 Stack Level' (see Subsection~\ref{ssec-over}). Note that calculation of
\r
941 character classes for each Japanese character is done \emph{after} the
\r
942 these callbacks for now.
\r
944 \subsection{Stack management}
\r
947 As we noted in Subsection~\ref{ssec-csname}, parameters that the values
\r
948 at the end of a horizontal box or that of a paragraph are effective in
\r
949 whole box or paragraph, such as \emph{kanjiskip}, cannot be implemented
\r
950 by internal integers or registers of other types in \TeX. We explain it
\r
955 void package(int c)
\r
961 if (cur_list.mode_field == -hmode) {
\r
962 cur_box = filtered_hpack(cur_list.head_field,
\r
963 cur_list.tail_field, saved_value(1),
\r
964 saved_level(1), grp, saved_level(2));
\r
965 subtype(cur_box) = HLIST_SUBTYPE_HBOX;
\r
968 \caption{An extract of a CWEB-source \texttt{tex/packaging.w} of \LuaTeX.}
\r
972 Figure~\ref{fig-ltsrc} is an extract of a CWEB-source
\r
973 \texttt{tex/packaging.w} of \LuaTeX\ (SVN revision 4358). This function
\r
974 is called just when an explicit |\hbox{...}| or |\vbox{...}| is ended, and
\r
975 the function |filtered_hpack()| is where the |hpack_filter| and then the
\r
976 actual `hpack' process are performed. Notice that the |unsave()|
\r
977 function is called before |filtered_hpack()|. This is the problem;
\r
978 because of |unsave()|, we can retrive only the values of registers
\r
979 \emph{outside} the box, even in the |hpack_filter| callback.
\r
981 To cope with this problem, \LuaTeX-ja has its own stack system, based on
\r
982 Lua codes in \cite{stack-mail}. Furthermore, \emph{whatsit} nodes whose
\r
983 \emph{user\_id} is 30112 (\emph{stack\_node}, for short) will be
\r
984 appended to the current horizontal list each time the current stack
\r
985 level is incremented, and their values are the values of
\r
986 |\currentgrouplevel| at that time. In the beginning of the |hpack_filter|
\r
987 callback, the list in question is traversed to determine whether the
\r
988 stack level at the end of the list and that outside the box coincides.
\r
990 Let $x$ be the value of |\currentgrouplevel|, and $y$ be the current
\r
991 stack level, both inside the |hpack_filter| callback, i.e.,~outside a
\r
992 horizontal box. Consider a list which represents the content of the box,
\r
995 \item A \emph{stack\_node} whose value is $x+1$ (because all materials in
\r
996 the box are included in a group |\hbox{...}|, the value is at
\r
997 least $x+1$) in the list represents an assignment related to the
\r
998 stack system in just top-level of the list, like
\r
1001 \hbox{...(assignment)...}
\r
1004 In this case, the current stack level is incremented to $y+1$ after the assignment.
\r
1005 \item A \emph{stack\_node} whose value is more than $x+1$ in the list represents
\r
1006 an assignment inside another group contained in the box. For example,
\r
1007 the following input creates
\r
1008 a \emph{stack\_node} whose value is $x+3=(x+1)+2$:
\r
1011 \hbox{...{...{...(assignment)}...}...}
\r
1015 Thus, we can conclude that the stack level at the end of the list is
\r
1016 $y+1$, if and only if there is a \emph{stack\_node} whose value is
\r
1017 $x+1$. Otherwise, the stack level is just $y$.
\r
1019 \subsection{Adjustment of the position of Japanese characters}
\r
1020 \label{ssec-width}
\r
1022 The size of a glyph specified in a metric and that of a real font
\r
1023 usually differ. For example, the letter `\inhibitglue【' is half-width
\r
1024 in |jfm-ujis.lua| or |jis.tfm|, while this letter is full-width like `【'
\r
1025 in most TrueType fonts used in Japanese typesetting, such as
\r
1026 IPA~Mincho. Hence the adjustment of position of such glyphs is
\r
1027 needed. In the context of \pTeX, this process was performed using virtual fonts.
\r
1029 On the other hand, Lua\TeX-ja does the adjustment by encapsuling a glyph
\r
1030 into a horizontal box. There are two main reasons why we adopted this
\r
1031 method; one is that we feared Lua codes for coexisting with callbacks by
\r
1032 |luaotfload| package would be large if we use virtual fonts, and the
\r
1033 other is to cope with shifting of the baseline of characters at the
\r
1037 \begin{center}\unitlength=9pt\small
\r
1038 \begin{picture}(15,12)(-1,-3)
\r
1040 \color{grayx}% real glyph
\r
1041 \put(-1,-1.5){\vrule width 6\unitlength height 7\unitlength depth 2.5\unitlength}
\r
1043 \color{black}% real glyph :step1
\r
1045 \put(-1,-1.5){\line(0,1){7}\line(0,-1){2.5}}
\r
1046 \put(5,-1.5){\line(0,1){7}\line(0,-1){2.5}}
\r
1047 \put(-1,5.5){\line(1,0){6}}
\r
1048 \put(-1,-4){\line(1,0){6}}
\r
1049 \put(-1,0){\makebox(0,0)[r]{\strut$R$\,}}
\r
1052 \put(0,0){\vector(0,1){9}\line(0,-1){3}\vector(1,0){12}}
\r
1053 \put(12,9){\makebox(0,0)[rt]{\strut$M$\,}}
\r
1054 \put(12,0){\line(0,1){9}\vector(0,-1){3}}
\r
1055 \put(0,9){\line(1,0){12}}
\r
1056 \put(0,-3){\line(1,0){12}}
\r
1057 \put(0.2,4.5){\makebox(0,0)[l]{\texttt{height}}}
\r
1058 \put(12.2,-1.5){\makebox(0,0)[l]{\texttt{depth}}}
\r
1059 \put(6,0.2){\makebox(0,0)[b]{\texttt{width}}}
\r
1062 \put(3,0){\line(0,1){7}\line(0,-1){2.5}\line(1,0){6}}
\r
1063 \put(9,0){\line(0,1){7}\line(0,-1){2.5}}
\r
1064 \put(3,7){\line(1,0){6}}
\r
1065 \put(3,-2.5){\line(1,0){6}}
\r
1066 \newsavebox{\eqdist}
\r
1067 \savebox{\eqdist}(0,0)[c]{%
\r
1069 \put(-0.08,0.2){\line(0,-1){0.4}}%
\r
1070 \put(0.08,0.2){\line(0,-1){0.4}}}
\r
1071 \put(1.5,0){\usebox{\eqdist}}
\r
1072 \put(10.5,0){\usebox{\eqdist}}
\r
1075 \put(3,-1.5){\vector(-1,0){4}}
\r
1076 \put(1,-1.7){\makebox(0,0)[t]{\texttt{left}}}
\r
1077 \put(3,0){\vector(0,-1){1.5}}
\r
1078 \put(3.2,-0.75){\makebox(0,0)[l]{\texttt{down}}}
\r
1081 \caption{The position of the `real' glyph.}
\r
1085 Figure~\ref{fig-pos} shows the adjustment process. A large square $M$ is
\r
1086 the imaginary body which is specified in the metric, and a vertical
\r
1087 rectangle is the imaginary body of a real glyph. First, the real glyph
\r
1088 is aligned with respect to the width of $M$. In the figure, the real
\r
1089 glyph is aligned `middle'; this setting is useful for the full-width
\r
1090 middle dot `・'. We have other settings, namely, `left' and `right'.
\r
1091 After that, it is shifted according to the value of |left| and |down|,
\r
1092 which are specified in the metric. The final position of the real glyph
\r
1093 is shown by the gray rectangle~$R$. If the amount of shifting the baseline is
\r
1094 not zero, $M$ (and hence the real glyph) is shifted by that amount.
\r
1096 We would like to remark briefly about the vertical position of a glyph.
\r
1097 A JFM (or the metric used in \LuaTeX-ja) and the real font used for it
\r
1098 may have different height or depth. In that case, it may look better if
\r
1099 the real glyph is shifted vertically to match the height-depth ratio
\r
1100 specified in the metric. This situation is carefully studied by
\r
1101 Otobe~\cite{min10}. Here the policy on this problem is not determined
\r
1102 now, however we would like to offer several solutions in future development.
\r
1104 \section{Conclusion}
\r
1105 We have discussed about our \LuaTeX-ja package, which is much affected
\r
1106 by \pTeX. For now, it can be used for experimental use, however there
\r
1107 are much refinements which are needed for regular use. The author hopes
\r
1108 that this paper and this project contribute the typesetting Japanese,
\r
1109 and possibly other Asian languages, under \LuaTeX.
\r
1111 \section*{Acknowledgements}
\r
1112 The author would like to thank Ken Nakano and Hideaki Togashi for their
\r
1113 development of ASCII \pTeX. The author is very grateful to Haruhiko
\r
1114 Okumura for his leadership in the Japanese \TeX\ community. The author
\r
1115 is also very grateful to members of \LuaTeX-ja project team for their
\r
1116 valuable cooperation in development.
\r
1118 %%% The style of the bibiliogrphy is `amsplain'.
\r
1119 \providecommand{\bysame}{\leavevmode\hbox to3em{\hrulefill}\thinspace}
\r
1120 \providecommand{\href}[2]{#2}
\r
1121 \begin{thebibliography}{99}
\r
1124 Adobe Systems Incorporated, \emph{Adobe-Japan1-6 Character Collection
\r
1125 for CID-Keyed Fonts}, Technical Note~\#5078, 2004.
\r
1126 \url{http://partners.adobe.com/public/developer/en/font/5078.Adobe-Japan1-6.pdf}
\r
1129 ASCII MEDIA WORKS，アスキー日本語\TeX\ (\pTeX)．\url{http://ascii.asciimw.jp/pb/ptex/}
\r
1132 Jin-Hwan~Cho and Haruhiko Okumura, \emph{Typesetting CJK Languages with Omega},
\r
1133 \TeX, XML, and Digital Typography, Lecture Notes in Computer Science, vol.~3130,
\r
1134 Springer, 2004, 139--148.
\r
1137 Yannis Haralambous. \emph{The Joy of \LuaTeX}. \url{http://luatex.bluwiki.com/}
\r
1139 \bibitem{jisx4051}
\r
1140 Japanese Industrial Standards Committee. \emph{JIS~X~4051: Formatting
\r
1141 rules for Japanese documents}, 1993, 1995, 2004.
\r
1144 北川弘典，$\varepsilon$-\pTeX についてのwiki．
\r
1145 \url{http://sourceforge.jp/projects/eptex/wiki/FrontPage}
\r
1148 北川弘典，\LuaTeX で日本語．
\r
1149 \url{http://oku.edu.mie-u.ac.jp/tex/mod/forum/discuss.php?d=378}
\r
1151 \bibitem{luatexref}
\r
1152 \LuaTeX\ development team, \emph{The \LuaTeX\ reference}.
\r
1153 \url{http://www.luatex.org/svn/trunk/manual/luatexref-t.pdf} (snapshot of SVN trunk)
\r
1156 \LuaTeX-ja project team, \emph{The \LuaTeX-ja package}.
\r
1157 Not completed for now. Available at |doc/man-en.pdf| (in English) or
\r
1158 |doc/man-ja.pdf| (in Japanese)
\r
1159 in the Git repository.
\r
1161 \bibitem{luajp-test}
\r
1162 香田温人，\LuaTeX と日本語．
\r
1163 \url{http://www1.pm.tokushima-u.ac.jp/~kohda/tex/luatex-old.html}
\r
1165 \bibitem{luajalayout}
\r
1166 前田一貴，luajalayout パッケージ---Lua\LaTeX によ
\r
1168 \url{http://www-is.amp.i.kyoto-u.ac.jp/lab/kmaeda/lualatex/luajalayout/}
\r
1170 \bibitem{jsclasses}
\r
1171 奥村晴彦，p\LaTeXe 新ドキュメントクラス．
\r
1172 \url{http://oku.edu.mie-u.ac.jp/~okumura/jsclasses/}
\r
1175 Haruhiko Okumura, \emph{\pTeX\ and Japanese Typesetting},
\r
1176 The Asian Journal of \TeX\ \textbf{2}~(2008), 43--51.
\r
1179 乙部厳己，min10フォントについて．
\r
1180 \url{http://argent.shinshu-u.ac.jp/~otobe/tex/files/min10.pdf}
\r
1183 齋藤修三郎，Open Type Font用VF．
\r
1184 \url{http://psitau.kitunebi.com/otf.html}
\r
1186 \bibitem{stack-mail}
\r
1187 Jonathan Sauer, \emph{[Dev-luatex] tex.currentgrouplevel}.
\r
1188 \url{http://www.ntg.nl/pipermail/dev-luatex/2008-August/001765.html}
\r
1191 Takuji Tanaka, \emph{u\pTeX, up\LaTeX---unicode version of \pTeX, p\LaTeX}.
\r
1192 \url{http://homepage3.nifty.com/ttk/comp/tex/uptex_en.html}
\r
1195 Nobuyuki Tsuchimura, \emph{Development of a Japanese \TeX\ Distribution~`ptetex3'},
\r
1196 Computer Software\ \textbf{24} (2007), no.~4, 40--50, (in Japanese).
\r
1199 W3C Working Group, \emph{Requirements for Japanese Text Layout}.
\r
1200 \url{http://www.w3.org/TR/jlreq/}
\r
1201 \end{thebibliography}
\r
OSDN Git Service
