The initial implementation of LSI [7] primarily was used by Bellcore as a research vehicle to test the LSI model and to discover any modifications that might have improved the model. The design of the initial implementation was weighted toward maximum flexibility rather than optimal execution efficiency.
While the initial implementation served as an excellent research tool, it has several shortcomings as a production-level system. Although most of the code that performs computation was written in the C programming language, UNIX (UNIX is a trademark of AT&T) shell scripting languages (Bourne shell and C shell) as well as the UNIX utilities awk and sed were used for text manipulation. The computational portions of the code were linked together with UNIX pipes, often with an awk or sed script sandwiched between them to convert the output of one computation into the form required for the next computation.
Besides the maintenance issues that arise when dealing with a single system consisting of so many different languages, efficiency issues become apparent for large document collections. For example, the time required to spawn several processes and establish pipes between them is significantly larger than the time required to begin executing a single, compiled program. In addition, the scripting languages as well as awk and sed are interpreted, resulting in even slower execution. Converting data formats between phases of execution and writing temporary files adds to the inefficiency.
A more subtle issue than either maintenance or efficiency is portability. While most platforms currently offer a means to compile C code, operating systems other than UNIX may not offer equivalent scripting languages and may not provide utilities like awk and sed. Most importantly, though, other operating systems may not offer pipes as a way to communicate information among separately executing processes.