Back to Contents List

Change History

Here is the change history of Detagger

Version 2.4 (June 2005)

Changes made in 2.4
Policies added in 2.4

Version 2.3.2 (September 2004)

Changes in version 2.3.2
New policies in version 2.3.2
Bugs fixed in version 2.3.2

Version 2.3 (April 2004)

Changes in Version 2.3
New policy options in Version 2.3

Version 2.2 (May 2003)

Changes in Version 2.2
New features in Version 2.2

Version 2.1 (March 2003)

Changes in Version 2.1
New features in Version 2.1

Version 2.0 (December 2002)

Changes in Version 2.0
New features in Version 2.0

Version 1.0 (August 2002)

Version 2.4 (June 2005)

Version 2.4 contains a small number of minor bug fixes and policy changes

Changes made in 2.4

• a new menu option Data Extraction appears under the Conversion Options -> Conversion to Text menu. These options allow you to select which data should be extracted from the soured document.

Policies added in 2.4

• Show only table data. The option is part of the new Data Extraction polices, and allows you to specify that only text contained in HTML <TABLE> markup should be included in the output document.
• Maximum table depth and Minimum table depth. These options are also part of the new Data extraction policies, and allow you to specify which levels of a set of nested tables should be included in the output.
• Ignore table WIDTH attributes. Sometimes the widths set in an HTML table don't give enough room to lay out the text when converting to text. In such cases this policy allows all WIDTH attributes to be ignored.
• The policy preserve hyperlinks in text output added in version 2.3.2 is now accessible on the Text Hyperlink policies option panel.
• The policies Remove HTML <DIV> tags and Remove HTML <SPAN> tags is added.
• The policy May break words to fit target width is added. This allows Detagger to break up long words in a table in order to allow the column to be made narrow enough to fit a small target width.
• The policy List item templates is added. This allows you to specify the text "decoration" added to ordered list items.
• The policy Nested Table scaling factor (percent) is added. This can be used to limit the space allowed to nested tables. This can assist when trying to squeeze heavily nested and wide tables into a limited fixed width.

Bugs fixed :-

• Text commands using match or replace expressions containing double quote characters weren't being accepted.
• Tables nested inside other tables where the outer table contained no <tr>..</tr> tags were not being processed correctly.
• Various policies were not being saved correctly after being changed, or when policies were saved to file.
• When Preserve all white space from the original source is enabled tabs in the original document are preserved. Previously they were being converted to spaces.
• A number of policies weren't being saved to policy file on exit, especially if users changes between tabs on the property sheet before pressing "Apply"
• PRE sections inside tables were being handled poorly. Large PRE sections (several 1000 lines) caused a performance hit which is now fixed, and in heavily nested tables the PRE text was compressed and lost it's formatting. Now the presence of a PRE section is liable to cause the table to become wider, and if the PRE section won't fit each line will be broken over several lines, but the formatting is preserved.

Version 2.3.2 (September 2004)

Version 2.3.2 contains a number of bug fixes and minor enhancements over version 2.3. It also contains enhanced support for handling Unicode, especially UTF-16.

Changes in version 2.3.2

• Improved Unicode support, and added support for UTF-16. See Working with Unicode
• New German translation of the user interface supplied by Jürgen Krane.
• Added support for DATA tag and IN_FILENAME and OUT_FILENAME attribute3 in text fragments. See The DATA fragment tag
• Changed the default page width to 76 from 72. For pages that add a <br> to each line this reduces the chances on unwanted additional line feeds in the output when converting to text. You can always override this value via the policy target page width
• Various formatting changes to improve table layout (especially nested tables).

New policies in version 2.3.2

Here are the policies added in version 2.3.2 :-

• Input text encoding
• Remove <!DOCTYPE> tags
• Preserve hyperlinks in text output

Bugs fixed in version 2.3.2

• Fixed a number of bugs when processing tables with missing <TR>, </TR> and <TD> tags. These could cause the table to not be processed or, in extreme cases, to be omitted from the output.
• Fixed bug where lines starting with a quotation mark didn't get correctly set to the target page width.
• Fixed bug whereby a nested table was indented when converted to delimited data.
• Fixed a number of bugs in which the presence of tags in embedded JavaScript (such as <style>, <body>, <title> etc) was confusing the file parsing, and occasionally leaving unwanted Javascript in the output.
• Files containing Unicode entities were not getting the Unicode file marker added when converted to text. This prevented some files (e.g. Arabic) from being displayed properly.
• A bug meant that occasionally the last line of a file wouldn't be read properly. This might have caused some conversion issues but was actually spotted when the last line in a policy file was ignored.
• Documents with a missing or misplaced </head> tag were converted to an empty file when converted to text.
• When removing hyperlinks, the first link after a NAME anchor tag wasn't being removed.
• When converting multiple files (as opposed to wildcards) the program would sometimes get confused calculation the output filename

Version 2.3 (April 2004)

Here are the policies added in version 2.3 :-

Changes in Version 2.3

• When converting files in sub-folder of the main input directory new options on the main screen allow you to have the output files placed in a parallel folder structure under the output folder. See Output Directory
• For the console version the /SUBFOLDERS and /TREE qualifiers are introduced to allow you to search sub-folders to locate input files, and to direct output files into a parallel folder structure to the input files.
• A new feature known as "Text Commands" allows you perform certain edits and manipulations on the source text before it is passed to the conversion process. For files that are not straight HTML files, but have instead come from elsewhere, these manipulations can be used to remove certain features (e.g. a non-HTML header on the document), or to turn certain markers into something that makes the file more HTML-like.

For example SEC filings use the HTML-like <TEXT> tag to markup plain text. By using a Text Command to change this into a <PRE> tag, the HTML converter is then tricked into leaving the format of the text in this section alone.

See Using a Text Commands File

• A new Tip of the Day feature has been added. The good news is that if you don't like it you can switch it off straight away and it will never bother you again :-)

Alternatively, the tips can be read in sequence should you prefer by using the next/last buttons to go through them, and the screen can be brought up from an option on the settings menu.

If anyone has suggestions as to topics they would like tips on, please feel free to send them to info<at>jafsoft.com.

• A Dutch translation has been supplied courtesy of Jurrien Dokter
  
  

New policy options in Version 2.3

Several new policy options are added in version 2.3

General

• Lines to ignore at end of file
• Lines to ignore at start of file

External configuration files

• Fragments File
• Text Commands File

Conversion to text

• Allow headings inside tables
• Default table indentation
• Target table width
• Heading underline characters
• Bullet point characters
• Text bullet characters
• Start list marker and End list marker
• Start table marker and End table marker
• Output indentation positions
• Allow by-line to be used for Author field

Markup removal

• Remove HTML table tags
• Remove HTML <OBJECT> tags

Version 2.2 (May 2003)

Version 2.2 contains a small number of improvements and enhancements over version 2.1.

Changes in Version 2.2

On the Settings menu a new option allows you to Remember settings on exit. If selected the current file, output directory, policy file and conversion options are remembered and used as the starting values next time you run the program.

New features in Version 2.2

A number of new options have been added to allow you to remove certain types of tags and attributes from inside tables only.

A new "Tables" option has been added under "markup manipulation" on the Conversion options menu. This takes you to the Detag Tables options tab which has the following options

• Remove HTML <P> tags from tables
  
• Remove HTML alignment attributes from tables
• Remove HTML color attributes from tables
• Remove HTML size attributes from tables

Version 2.1 (March 2003)

Version 2.1 contains a small number of improvements and enhancements over version 2.0.

Changes in Version 2.1

• Performance improvements. A number of small bugs have been fixed. Also a number of performance enhancements have been made. These will only really be noticeable when converting large files.

New features in Version 2.1

• Concatenate results. It is now possible to convert several files at once and have the results output to a single file. To do this select "Concatenate results into a single file" for the Conversion Type.
• Searching sub-folders. It is now possible to get the program to look in sub-folders of the one selected for files to convert. To do this just tick the "Search Sub-folders" checkbox. Note, at present all results will only be put in a single output folder, and not into a structure that matches the original. See Input selection
• New Table border option. A new option allows you to output blank lines between rows when converting tables to text. See Allow blank row separator lines
• New table data handling options. New options allow you to fine tune the conversion of tables to delimited data. See Convert only innermost tables and Add delimited table markers
  
  

Version 2.0 (December 2002)

Version 2.0 contains a number of changes suggested by users of Detagger, as well as a number of bug fixes and code enhancements.

Changes in Version 2.0

• Name change. I've decided to lose the "-" in the name, so "De-tagger" becomes "Detagger" as of version 2.0 :-)
• The version 2.0 code has been improved when dealing with large files Previously the performance seriously degraded on files over 0.5 Mb in size. The new version is significantly faster when handling such files, and furthermore there is now no theoretical limit to file size, although if a whole file is placed in a table there may still be problems.
• Several bugs with saving options ("policies") to file have been fixed. The main dialog now has a section for policy files to be loaded to make their use much more convenient. Recently saved files can be quickly recalled from the drop down list.
• A console version is now available to registered users. The console version can be run from the command line and is better suited to batch operation. A demonstration of the console version can be got from the website at http://www.jafsoft.com/developers/console_demos.html
• For those wishing to integrate the functionality of Detagger into their own software, and API version is available. This is sold under a separate license, subject to negotiation based on the intended use of the API. You can find out more and download an evaluation version from http://www.jafsoft.com/developers/api_demos.html
  
  

New features in Version 2.0

Several new features have been added to Detagger since version 1.0.

Markup removal

Tag removal options

Remove emphasis tags
Removes all the bold and italic markup from the HTML

Remove style sheet
Removes all the <STYLE> sections from the HTML, together with any reference to an external CSS style sheet.

Remove HTML <IMG> tags
Removes all the <IMG> tags from a HTML document.

HTML-to-Text conversion

Paragraph formatting

Output each paragraph on a single line
Each paragraph is output without hard line breaks (except at the end). This can be useful, depending on how and where the resulting text is to be used

Miscellaneous text formatting

May add Unicode marker to output file
When Unicode is detected in the source the software will output the text as UTF8 and optionally add a file marker that will label the file as "Unicode" in a way that most applications that can cope with Unicode will recognize.

Hyperlinks handling

Display link URLs
An option to display hyperlink URLs immediately after the display text in the output.

Replace <IMG> tags by a text marker
Option to place a marker in the text to show where an image has been removed.

Use the ALT attribute to replace <IMG> tags
Option to use the ALT attribute of an <ING> tag in it's text marker. This can help give some sense of what was being shown on the original page.

Tables conversion

Convert table to plain text
Convert table to comma-delimited data
Convert table to tab-delimited data
These options (which are mutually exclusive), determine how any <TABLE>s in the HTML should be output in the text. The options allow the table to be converted to plain text, or to delimited text better suited for loading into a spreadsheet such as Excel.

Version 1.0 (August 2002)

The initial release.

Back to Contents List

Converted from a single text file by AscToHTM © 1997-2005 John A Fotheringham