TF: Textfile & String Library for AutoHotkey [lib] - v3.4

A "Swiss Army Knife" library for Text (files)

Introduction

As the name suggest this is an AutoHotkey (AHK) Library with a number of functions to "manipulate" text, both files such as *.txt, *.ahk, *.html, *.css etc AND Strings (or variables). For example you can delete specific lines, replace words or specific lines, number lines, remove or insert columns of text, etc. See the list of functions above for a complete overview.

It is not useful for binary files or data such as MS Office files, PDFs, EXEcutables, images etc. (Tip: search the AHK forum for "binread" to find some pointers on how to read, write and "edit" binary files.)

Functions

TF
TF_Save
TF_CountLines
TF_Count *1
TF_ReadLines
TF_Tail
TF_Replace
TF_ReplaceInLines
TF_RegExReplace
TF_RegExReplaceInLines
TF_RemoveLines
TF_RemoveBlankLines
TF_RemoveDuplicateLines
TF_InsertLine
TF_ReplaceLine
TF_InsertPrefix
TF_InsertSuffix
TF_TrimLeft
TF_TrimRight
TF_AlignLeft
TF_AlignCenter
TF_AlignRight
TF_LineNumber
TF_ConCat
TF_ColGet
TF_ColPut
TF_ColCut
TF_ReverseLines
TF_Find (can be used as basic grep)
TF_SplitFileByLines
TF_SplitFileByText
TF_Merge *2
TF_Prepend *2
TF_Append *2
TF_Wrap
TF_WhiteSpace
TF_Substract
TF_RangeReplace
TF_MakeFile
TF_Tab2Spaces
TF_Spaces2Tab
TF_Sort
TF_Join

Notes

TF_Count does not support files and only works with variables.
TF_Merge, TF_Prepend, TF_Append currently do not support variables and only work with FILES.

Concept

It is important to understand a few basic concepts before you start working with the TF library:

If you pass on a file to a TF function the output will generally be a file (there are a few exceptions). By default it writes the output to a COPY of the input file, leaving the original input file intact.
You can overwrite the input file by using a ! as prefix. See "Quick intro to Parameters" and "Textfile and the ! Prefix" below for more details.
If you pass on a variable to a TF function the returned output will also be a variable.
Most TF Functions allow you to work on specific lines or sections of lines.
TF "knows" if something is a file or a variable, even if the variable passed on to TF is meant to represent a file. (This is done in the helper function TF_GetData for those interested).

To understand how to work with files and variables please read the "Textfile and the ! Prefix" and "Files & Variables" sections below.

File encoding, codepage

If you experience TF is changing the file encoding (codepage) try to set the correct file encoding at the top of your script (or at least before you call a TF function) using the following AutoHotkey command: http://l.autohotkey.net/docs/commands/FileEncoding.htm

Please note, there is no way to determine the file encoding 100% accurately, even if a file contains BOM. See the discussion about this topic and possibly useful additional functions at the AutoHotkey forum http://www.autohotkey.com/board/topic/95986-filegetencoding-filegetformat/ (The result of these functions are simply a best guess assuming UTF-8 more common when BOM - https://en.wikipedia.org/wiki/Byte_Order_Mark - is missing).

How to "install" / use

You can either place TF.ahk in your LIB directory (see http://www.autohotkey.com/docs/Functions.htm#lib) or use #include (see http://www.autohotkey.com/docs/commands/_Include.htm)

You can find examples of most functions in the "example script" here http://www.autohotkey.com/forum/viewtopic.php?p=280363#280363

Note: Because most of these functions operate on a line by line basis they WILL be slower than if you write your own functions or script which could operate on an entire file or variable at once. Keep this in mind if you need to process many files/variables or very large files/variables in case speed is an issue.

Quick intro to Parameters

Almost all functions accept the following basic parameters:

Parameter	Meaning
Text	The Filename (may include (absolute) path) to read from and save to (for all functions that write an output file). Note: by default a filename_copy will be created, use the ! prefix if you want to OverWrite the TextFile (e.g. the source file) As of v3 "Text" can also be a variable or indeed text directly passed on to the function. See Textfile and the ! Prefix.
! Prefix	If Text starts with ! (eg: "!c:\sample.txt") it will overwrite the text file, otherwise it will save the new file to a copy of the text file eg: Filename_copy.txt (All, apart from reading functions because there is no output file) Tip: you can use concatenation to add the !, e.g. "!" . Filename.txt, see the examples in the AHK thread. Note: If Text is a variable, it can start with a ! as TF will detect automatically that it is not a file and will therefore not create a file but return the altered variable instead. See Textfile and the ! Prefix.
Lines	Number of lines to read
StartLine	Start of Range (Almost all). See (StartLine, Endline) Syntax.
EndLine	End of Range (Almost all). See (StartLine, Endline) Syntax.
SearchText	Text to Find (Find / Replace functions)
ReplaceText	Text to Replace (Find / Replace functions)

Notes

Many functions also have specific parameters, see the description of each function below.
Backup files: If a subdirectory "backup" is present in the directory of TextFile a backup is made before overwriting the original file (both for file.txt and file_copy.txt) with the BAK extension
You can find examples of most functions in the "example script" here http://www.autohotkey.com/forum/viewtopic.php?p=280363#280363

(StartLine, Endline) Syntax

You can pass on multiple lines (sections) by using the StartLine parameter.

Examples:

(5) ; start from line 5 to the end (StartLine)
(5, 15) ; Section: lines 5 to 15 (StartLine, EndLine)
("5,13,45,67", 135) ; Specific lines: 5,13,45,67, ignore 135 ("StartLine", Endline - EndLine value is ignored)
("5-13,45-51", 135) ; Multiple sections of lines: 5 to 13, 45 to 51 ignore 135 ("StartLine", Endline - EndLine value is ignored)

Incremental (EndLine will not be ignored) in the following cases:

("2+3", 150) ; start with line 2, increment 3 up-to line 150 so 2,5,8,11,14 etc
("5+15") ; start with line 5, increment 15 until end of file so 5,20,35 etc

Negative Startline

A Negative startline value makes a TF function operate on the last X lines.
TF_ColGet and TF_ColPut accept negative StartColumn parameter. This can be used to "Get" or "Put" text in the x-th "column" from the right

Textfile and the ! Prefix

Valid examples for using a Textfile:

TF_("file.txt", .... ; Process file.txt and write output to file_copy.txt

F=file.txt ; ; Note how F is a variable, but AHK/TF will see it is meant to represent file.txt and write output to file_copy.txt
TF_(F, .... ; Process file.txt and write output to file_copy.txt
TF_("!file.txt", .... ; Process file.txt and overwrite file.txt

F=file.txt
TF_("!" . F, .... ; Process file.txt and overwrite file.txt

F=!file.txt
TF_(F, .... ; Process file.txt and overwrite file.txt

In a Loop, FilePattern:
TF_(A_LoopFileName, .... ; Process file and write output to file_copy
TF_("!" . A_LoopFileName, .... ; Process file and overwrite file

Loop, *.txt
    {
     TF_(A_LoopFileName, .... ; Process file and write output to file_copy
    }

If you want to use multiple TF functions on a single file it is advised to use the ! Prefix

F=!file.txt 
TF_RemoveBlankLines(F)        ; remove all empty lines from file.txt and overwrite original file.txt
TF_LineNumber(F)              ; add linenumbers to all lines and overwrite original file.txt
TF_Replace(F, "this", "that") ; Replace the word "this" with "that" and overwrite original file.txt
; So the original file.txt has undergone three changes

Files & Variables (v3+)

You can use TF functions directly on files as well as variables. Below you will find some easy examples.

Introduced in v3: TF()

In order to save you the trouble of using a fileread to read a file into a variable and proceed to use various TF functions there is the TF() Function.

TF("pathtofile", CreateGlobalVar="T")

By default it reads the contents of the file in a global variable named t and returns it for further processing. By default it creates variable t if a variable name is omitted so if you do not like to use t you can use anything you like. It is NOT mandatory to use TF(), you are welcome to use a fileread as well, TF() is merely added for convenience as we will see in the examples below. TF() doesn't suddenly make the TF library "not standard compliant" as some seem to think as it only creates a global variable if you use the TF() function.

Repeat: You only have to use TF() if you want to read a file into a variable and use it on multiple TF_* Functions but it is NOT required.

TF() Examples:

TF("file.txt") ; will create global var t
TF("file.txt", "MyVar") ; will create global var MyVar

Examples on how to use files & variables with TF:

#Include tf.ahk
TestFile= ; create variable
(join`r`n
1 Hi this
2 a test variable
3 to demonstrate
4 how to 
5 use this
6 new version
7 of TF.AHK
)
FileDelete, TestFile.txt
FileAppend, %TestFile%, TestFile.txt ; create file
F=TestFile.txt ; just a shorthand code for TextFile.txt, so when
; we are using 'F' below we are still passing on a TextFile, not a variable!

; pass on file, read lines 5 to end of file:
MsgBox % "From File 1:`n" TF_ReadLines("TestFile.txt",5)
MsgBox % "From File 2:`n" TF_ReadLines(F,5)              ; same

; pass on variable, read lines 1 to 5
MsgBox % "From Variable 1:`n" TF_ReadLines(TestFile,"1-5")     
MsgBox % "From Variable 2:`n" TF_ReadLines("Hi`nthis`nis`na`ntest`nvariable`nfor`ntesting",1,3) ; pass on string

; Examples using TF(): (it will save you a FileRead if you want to work with Variables)

TF("TestFile.txt") ; read file, create global var t
t:=TF_ReadLines(t,5) ; pass on global var t created by TF(), read lines 5 to end of file, assign result to t 
MsgBox % "TF(), example 1:`n" t

TF("TestFile.txt", "MyVar") ; read file, create global var MyVar
MyVar:=TF_ReadLines(MyVar,5) ; pass on global var MyVar created by TF(), read lines 5 to end of file, assign result to MyVar
MsgBox % "TF(), example 2:`n" MyVar

; Note how we can use TF() here
t:=TF_ReadLines(TF("TestFile.txt"),5) ; pass on global var t created by TF(), read lines 5 to end of file, assign result to t 
MsgBox % "TF(), example 3:`n" t

MyVar:=TF_ReadLines(TF("TestFile.txt","MyVar"),5) ; pass on global var t created by TF(), read lines 5 to end of file, assign result to t 
MsgBox % "TF(), example 4:`n" MyVar

t:=TF_ReadLines(TF(F),5) ; pass on global var t created by TF(), read lines 5 to end of file, assign result to t
t:=TF_ReverseLines(t,5) ; pass on global var t created by TF(), reverse lines, assign result to t
MsgBox % "TF(), example 5:`n" t

; Work directly with the clipboard or another other variable
Clipboard=Line 1`nLine 2`nLine 3`nLine 4`nLine 5`nLine 6`nLine 7`nLine 8
Clipboard:=TF_RemoveLines(Clipboard, 3, 6) ; remove lines 3 to 6
MsgBox % "Clipboard, example 6:`n" Clipboard

Note TF_Merge, TF_Prepend, TF_Append currently do not support variables and only work with FILES.

Common mistake(s):

MyVar:=TF_ReadLines("TestFile.txt",5) ; this is wrong. It is AN INCORRECT EXAMPLE!

The example above is incorrect: You pass on a file so the output is testfile_copy.txt. In this case nothing meaningful is assigned to the variable MyVar. Correct would be:

MyVar:=TF_ReadLines(MyVar,5) ; this is OK
MyVar:=TF_ReadLines(TF("TestFile.txt","MyVar"),5) ; this is OK

Functions and Examples

Note: Some of these functions are very similar, but they are included in the library for ease of use ...

You can find examples of most functions in the "example script" here http://www.autohotkey.com/forum/viewtopic.php?p=280363#280363

TF(TextFile, CreateGlobalVar="T")

Purpose: Read contents of file to create global variable, T by default.
Parameters: TextFile, CreateGlobalVar (=preferred name of global variable)
Credits: See TF thread & discussion http://www.autohotkey.com/forum/viewtopic.php?p=313120#313120

Note: see background info and examples above at Files and Variables.

TF_CountLines(Text)

Purpose: Returns the number of lines in a file or variable
Parameters: Text

Credits: SKAN

MsgBox % TF_CountLines("File.txt") ; show the number of lines of file in a MsgBox
Lines:=TF_CountLines("File.txt") ; store the number of lines of file in a variable

TF_Count(String, Char)

Purpose: Count the number of occurrence of Char using StringReplace
Parameters: String, Char

Notes: do not use it to count lines (using `n) because it will return an incorrect value, use TF_CountLines for counting lines. TF_Count does not support files and only works with variables.

MsgBox % TF_Count("Hello this is an example", "i") ; count how many i's there are in the string

TF_ReadLines(Text, StartLine = 1, EndLine = 0, RemoveTrailing = 0)

Purpose: Return the specified lines, can be used to read sections of a file or variable.
Parameters: Text, StartLine, EndLine

Note: by default it will add a newline character to the last line, so if you want to append new data you don't have to start it with a newline char. If you don't want that use RemoveTrailing = 1.

MsgBox % TF_ReadLines("File.txt",5) ; Read lines 5 to end of file, show result in a MsgBox
Lines:=TF_ReadLines("File.txt",5) ; Read lines 5 to end of file, store result in variable
MsgBox % TF_ReadLines("File.txt",5,0,1) ; 0 for end line indicates until end of file, remove trailing empty line.

TF_Tail(Text, Lines = 1, RemoveTrailing = 0, ReturnEmpty = 1)

Purpose: Read last X lines from file or variable
Parameters: Text, Lines, RemoveTrailing (0 or 1), ReturnEmpty (0 or 1)
Credits: incl. ideas from Tuncay

Notes:

Lines can be a negative number: -1 will get the second to last line. If you use a negative number it will ALWAYS retrieve a SINGLE line
By default TF_Tail will return empty lines, if you don't want that use ReturnEmpty = 0
By default TF_Tail will add a newline character to the last line, so if you want to append new data to the returned data you don't have to start it with a newline char. If you don't want that use RemoveTrailing = 1

MsgBox % TF_Tail("File.txt", 3) ; get the last three lines
MsgBox % TF_Tail("File.txt", -2) ; get second to last line, negative values only return one line
MsgBox % TF_Tail("File.txt", 5, 0, 0) ; return the last five lines, with trailing new line and excluding empty lines

TF_Replace(Text, SearchText, Replacement="")

Purpose: Find and Replace text in entire file (using StringReplace)

Parameters: Text, SearchText, Replacement

TF_Replace("File.txt","key","lock")            ; pass on a file, replace the word "key" with "lock" in file_copy.txt

TF_ReplaceInLines(Text, StartLine = 1, EndLine = 0, SearchText = "", ReplaceText = "")

Purpose: Find and Replace text on specific lines (using StringReplace)

Parameters: Text, StartLine, EndLine, SearchText, ReplaceText

TF_ReplaceInLines("!File.txt","1,3,9","","key","lock") ;  update source file, replace "key" with "lock" in lines 1, 3 and 9

TF_RegExReplace(Text, NeedleRegEx = "", Replacement = "")

Purpose: Find and Replace text in entire file (using RegExReplace)

Parameters: Text, NeedleRegEx, Replacement

TF_RegExReplace("File.txt","im)^(.*)$","[$1]") ; pass on a file, wrap all lines in []

TF_RegExReplaceInLines(Text, StartLine = 1, EndLine = 0, NeedleRegEx= "", Replacement = "")

Purpose: Find and Replace text in specific lines (using RegExReplace)

Parameters: Text, StartLine, EndLine, NeedleRegEx, Replacement

TF_RegExReplaceInLines("File.txt",3,8," [a-z]{3} "," lock ")  ; replace any three letter word with "lock" in lines 3 to 8

TF_RemoveLines(Text, StartLine = 1, EndLine = 0)

Purpose: Remove specified lines from file

Parameters: Text, StartLine, EndLine

TF_RemoveLines("File.txt", 5, 10) ; remove lines 5 to 10 from file

Note: If you pass on a negative value for StartLine, example TF_RemoveLines(Text, -5, .... it will remove these lines from the end of Text, so -5 will remove the last five lines. The EndLine parameter is ignored.

TF_RemoveBlankLines(Text, StartLine = 1, EndLine = 0)

Purpose: Remove blank lines from file (in specified section)
Parameters: Text, StartLine, EndLine

Note: also removes lines with only tabs & spaces e.g. "white space"

TF_RemoveBlankLines("File.txt") ; remove blanklines in entire file

TF_RemoveDuplicateLines(Text, StartLine = 1, Endline = 0, Consecutive= 0, CaseSensitive = false)

Purpose: Remove duplicate lines, optionally you can specify if they need to be Consecutive and / or Case sensitive

Parameters: Text, StartLine, EndLine, Consecutive (0 or 1), CaseSensitive (true or false)

TF_RemoveDuplicateLines("File.txt","","",1,false) ; remove only Consecutive duplicate lines

TF_InsertLine(Text, StartLine = 1, Endline = 0, InsertText = "")

Purpose: Insert text in specified lines

Parameters: Text, StartLine, Endline, InsertText

TF_InsertLine("File.txt","2,4,9",5,"---")     ; insert --- in lines 2 4 and 9. 5 is endline will be ignored

TF_ReplaceLine(Text, StartLine = 1, Endline = 0, ReplaceText = "")

Purpose: Replace specified lines with new text

Parameters: Text, StartLine, Endline, ReplaceText

TF_ReplaceLine("File.txt","1+3",8,"---")      ; replace lines 1 4 and 7. 8 is end line so no more lines are processed after

TF_InsertPrefix(Text, StartLine = 1, EndLine = 0, Text = "")

Purpose: Insert a text at the BEGINNING of each of the specified lines

Parameters: Text, StartLine, Endline, Text

TF_InsertPrefix("File.txt",5,8, "Hello ")          ; Prefix Hello in lines 5 to 8

TF_InsertSuffix(Text, StartLine = 1, EndLine = 0 , Text = "")

Purpose: Append a text at the END of each of the specified lines

Parameters: Text, StartLine, Endline, Text

TF_InsertSuffix("File.txt","2-4,6-9","", " Hello")  ; Suffix Hello in lines 2 to 4 and 6 to 9

TF_TrimLeft(Text, StartLine = 1, EndLine = 0, Count = 1)

Purpose: Trim number of specified columns from specified lines

Parameters: Text, StartLine, Endline, Text, Count

TF_TrimLeft("File.txt","","",25)                    ; Trim Left 25 Characters of all lines

TF_TrimRight(Text, StartLine = 1, EndLine = 0, Count = 1)

Purpose: Trim number of specified columns from specified lines

Parameters: Text, StartLine, Endline, Text, Count

TF_TrimRight("File.txt","4,6,8","",45)              ; Trim Right 45 Characters in lines 4, 6 and 8

TF_AlignLeft(Text, StartLine = 1, EndLine = 0, Columns = 80, Padding = 0)

Purpose: Align lines according to number of specified columns
Parameters: Text, StartLine, Endline, Columns, Padding
incl. ideas from SKAN

Note: Using the Align functions will not take into account any leading or trailing spaces a line had BEFORE processing. So all white space before and after the text is removed before the text is aligned.

Padding = 0 Remove trailing white space
Padding = 1 Keep trailing white space

TF_AlignLeft("File.txt","","",90, 1)    ; AlignLeft all lines, keep trailing white space

TF_AlignCenter(Text, StartLine = 1, EndLine = 0, Columns = 80, Padding = 0)

Purpose: Align lines according to number of specified columns
Parameters: Text, Columns, StartLine, Endline, Text
uses some of SKAN functions

Note: Using the Align functions will not take into account any leading or trailing spaces a line had BEFORE processing. So all white space before and after the text is removed before the text is aligned.

Skip = 0 process empty lines, fill with spaces
Skip = 1 skip empty lines, do not fill with spaces

TF_AlignCenter("File.txt","","",150, 1) ; AlignCenter, all lines skip empty lines, do not fill with spaces

TF_AlignRight(Text, StartLine = 1, EndLine = 0, Columns = 80, Skip = 0)

Purpose: Align lines according to number of specified columns
Parameters: Text, Columns, StartLine, Endline, Skip (0 or 1)
uses some of SKAN functions

Note: Using the Align functions will not take into account any leading or trailing spaces a line had BEFORE processing. So all white space before and after the text is removed before the text is aligned.

Skip = 0 process empty lines, fill with spaces
Skip = 1 skip empty lines, do not fill with spaces

TF_AlignRight("File.txt","","", 190, 0) ; AlignRight, all lines, do not skip empty lines fill them with spaces

TF_LineNumber(Text, Leading = 0, Restart = 0, Char = 0)

Purpose: Insert line numbers before each line
Parameters: Text, Leading, Restart, Char
Credits: incl. ideas from ribbet.1

Leading = 0 No padding with leading zeros
Leading >= 1 Include padding with leading zeros (001 v 1)
Restart = restart counter after X lines (starting again at 1)
Char = character to use for leading/padding. Default 0, but can be any character. Tip: use a space.

TF_LineNumber("File.txt",1,15,A_Space)  ; Add linenumbers, padding with spaces, linenumbers are aligned right because of space

TF_ConCat(FirstTextFile, SecondTextFile, OutputFile, Blanks = 0, FirstPadMargin = 0, SecondPadMargin = 0)

Purpose: Join Text files side by side (line1-from-file1 line1-from-file2 newline line2-from-file1 line2-from-file2 newline etc)
Parameters: FirstTextFile, SecondTextFile, OutputFile, Blanks, FirstPadMargin, SecondPadMargin

Based on: CONCATenate text files, ftp://garbo.uwasa.fi/pc/ts/tsfltc22.zip

With TF_ConCat you can join Text files side by side. Blanks is number of blanks between lines. You can pad blanks the right margin of either of the text files, for this use FirstPadMargin and SecondPadMargin.

TF_ColGet(Text, StartLine = 1, EndLine = 0, StartColumn = 1, EndColumn = 1, Skip = 0)

Purpose: Get specified columns from text file
Parameters: Text, StartLine, EndLine, StartColumn, EndColumn, Skip (0 or 1)

Note: A TAB character is 1 character so for files with TABS the column might not be where you expect it to be
skip = 0, DO NOT skip lines shorter then startcolumn position
skip = 1, skip lines shorter then startcolumn position
TF_ColGet and TF_ColPut accept negative StartColumn parameter. This can be used to "Get" or "Put" text in the x-th "column" from the right

TF_ColGet("File.txt","","",2,13)    ; get columns 2 to 13 from all lines, so it removes all other columns from the file or variable

TF_ColPut(Text, Startline = 1, EndLine = 0, StartColumn = 1, Text = "", Skip = 0)

Purpose: Insert text at specified columns in text file
Parameters: Text, StartLine, EndLine, StartColumn, EndColumn, Skip (0 or 1)

Based on: COLPUT.EXE & CUT.EXE, ftp://garbo.uwasa.fi/pc/ts/tsfltc22.zip

Note: A TAB character is 1 character so for files with TABS the column might not be where you expect it to be
skip = 0, DO NOT skip lines shorter then startcolumn position
skip = 1, skip lines shorter then startcolumn position
TF_ColGet and TF_ColPut accept negative StartColumn parameter. This can be used to "Get" or "Put" text in the x-th "column" from the right

TF_ColPut("File.txt","","",5, "|", 0) ; insert | in column 5 of all lines including empty lines (= | will be at column 1)

TF_ColCut(Text, StartLine = 1, EndLine = 0, StartColumn = 1, EndColumn = 1)

Purpose: Remove specified columns from text file
Parameters: Text, StartLine, EndLine, StartColumn, EndColumn

Based on: COLPUT.EXE & CUT.EXE, ftp://garbo.uwasa.fi/pc/ts/tsfltc22.zip

TF_ColCut("File.txt", "2+2", "", 4, 38)  ; remove columns 4 to 38 in lines 2 4 6 8 etc

TF_ReverseLines(Text, StartLine = 1, EndLine = 0)

Purpose: Reverse the order of specified lines
Parameters: Text, StartLine, EndLine

Note: Startline parameter can not use specific lines, sections or incremental values here

TF_ReverseLines("File.txt",2,9) ; reverse lines 2 to 9

TF_Find(Text, StartLine = 1, EndLine = 0, SearchText = "", ReturnFirst = 1, ReturnText = 0)

Purpose: Find text using RegExMatch, return line(s), text or lines and text
Parameters: Text, StartLine, EndLine, SearchText, ReturnFirst, ReturnText

TF_Find(Lines) uses Regular Expressions - This means that if you use certain characters which have a special meaning in a RegEx: \.*?+[{|()^$ they must be preceded by a backslash to be seen as literal. For example, \. is a literal period and \\ is a literal backslash. Escaping can be avoided by using \Q...\E. For example: \QLiteral Text\E. See http://www.autohotkey.com/docs/commands/RegExMatch.htm for further information.

ReturnFirst = 0 return multiple lines 
ReturnFirst = 1 return first line only

ReturnText = 0 return line numbers only
ReturnText = 1 return entire line (text). This simulates a basic grep feature
ReturnText = 2 return line numbers + entire line (text). This simulates a basic grep feature

MsgBox % TF_Find("File.txt", "", "", "keys") ; return first line number with keys in it
MsgBox % TF_Find("File.txt", "", "", " c[a-z]+s ", 0, 1) ; find all lines with words that start with a c an end with an s

TF_SplitFileByLines(Text, SplitAt, Prefix = "file", Extension = "txt", InFile = 1)

Purpose: Split a text file in to several others based on number of lines
Parameters: Text, SplitAt (number), Prefix, Extension, InFile (0, 1 or 2)

SplitAt = Number of lines (three methods, see below)
Prefix . Extension = filenameAUTOINCREMENT.Extension (Example: part_ . txt)
InFile = 0 skip line e.g. do not include the actual line in any of the output files
InFile = 1 include line IN current file
InFile = 2 include line IN next file

Note: If you pass on a variable to TF_SplitFileByLines, the array size will be returned as Prefix0 and the array elements as Prefix1, Prefix2 etc similar to the AHK StringSplit command. The Extension parameter is ignored when using variables. Some characters are illegal such as - + @ % & * _ \ / [ ] etc to use in Prefix, stick to a-z A-Z.

Options for "SplitAt" parameter:

a) One numerical value, example TF_SplitFileByLines(Text, "25", .... 
   will split text every 25 lines until the end 
b) Split at rotating line lengths using a dash "-" as separator, example TF_SplitFileByLines(Text, "5-10-15", .... 
   will split text at 5 10 15 lines, until the end so 5 lines, 10 lines, 15 lines, 5 lines, 10 lines etc
c) Split at specific lines using a comma "," as separator, example TF_SplitFileByLines(Text, "5,81,135", .... 
   will split text at lines 5,81,135 until end of file (e.g. last file will be from line 135 until the end)

TF_SplitFileByLines("File.txt", 2, "part", "zec", 1) ; split source file every 2 lines, include 2nd line INFILE
; illustrate use of variables and returned arrays:
TF_SplitFileByLines(Variable, 2, "part", "zec", 1)
MsgBox % "Array size: " . Part0 . "`n1st array element: "  Part1

TF_SplitFileByText(Text, SplitAt, Prefix = "file", Extension = "txt", InFile = 1)

Purpose: Split a text file in to several others based on text
Parameters: Text, SplitAt (text, can be RegEx), Prefix, Extension, InFile (0, 1 or 2)

SplitAt = Text, can be RegEx
Prefix . Extension = filenameAUTOINCREMENT.Extension (Example: part_ . txt)
InFile = 0 skip line e.g. do not include the actual line in any of the output files
InFile = 1 include line IN current file
InFile = 2 include line IN next file

Note: If you pass on a variable to TF_SplitFileByLines, the array size will be returned as Prefix0 and the array elements as Prefix1, Prefix2 etc similar to the AHK StringSplit command. The Extension parameter is ignored when using variables. Some characters are illegal such as - + @ % & * _ \ / [ ] etc to use in Prefix, stick to a-z A-Z.

TF_SplitFileByText("File.txt", "button", "part", "zec", 1)      ; split source file on every line with the word button, include that line INFILE
; illustrate use of variables and returned arrays:
TF_SplitFileByText(Variable, "keyboard", "part", "zec", 1)
MsgBox % "Array size: " . Part0 . "`n1st array element: "  Part1

TF_Merge(FileList, Separator = "`n", FileName = "merged.txt")

Purpose: Merge several files into one
Parameters: FileList, Separator (what to put between two files, newline by default), FileName (name of output file, Prefix with a ! to overwrite target file)

Example FileList:

FileList=
(
file1.txt
file2.txt
file3.txt
)

Note: TF_Merge, TF_Prepend, TF_Append currently do not support variables and only work with FILES.

; using Loop (files & folders) to create one quickly if you want to merge all TXT files for example:
Loop, c:\*.txt
  FileList .= A_LoopFileFullPath "`n"
TF_Merge(FileList) ; will create merged.txt, you can use ! to overwrite an existing file if you want

; using FileSelectFile to select files to merged: (Thanks for asking Vitor, http://www.autohotkey.com/forum/viewtopic.php?p=335329#335329)
FileDelete merged.txt ; not required
FileList=
FileSelectFile, FileList, M 1,,, *.txt ; M allows you to select multiple files while holding down the left ctrl button
If (ErrorLevel = 1) or (FileList = "")
   ExitApp ; no files selected
Path:=TF_ReadLines(FileList,1,1,1) ; the first line holds the directory of the selected files, so read path
FileList:=TF_RemoveLines(FileList,1,1) ; remove path from filelist
FileList:=TF_InsertPrefix(FileList, "", "", Path . "\") ; make sure all files have full paths to file so the are read correctly
TF_Merge(FileList) ; will create a file in the current script dir called merged.txt you can also specify another filename, take into account the filedelete merged.txt above

; You could skip the Path:= step above by calling TF_ReadLines directly in TF_InsertPrefix, but you would have to delete the first line AFTER it like so:
;FileList:=TF_InsertPrefix(FileList, "", "", TF_ReadLines(FileList,1,1,1) . "\")
;FileList:=TF_RemoveLines(FileList,1,1)

TF_Prepend(File1, File2)

Purpose: Prepend file1 to file2 (file2 is changed, uses TF_Merge)
Parameters: File1, File2

Note: TF_Merge, TF_Prepend, TF_Append currently do not support variables and only work with FILES.

TF_Append(File1, File2)

Purpose: Append file1 to file2 (file2 is changed, uses TF_Merge)
Parameters: File1, File2

Note: TF_Merge, TF_Prepend, TF_Append currently do not support variables and only work with FILES.

TF_Wrap(Text, Columns = 80, AllowBreak = 0, StartLine = 1, EndLine = 0)

Purpose: Wrap (specified) lines
Parameters: Text, Columns, AllowBreak (0 or 1), StartLine, EndLine

AllowBreak = 0 will not "break" words, so it will take into account whole words and not chop them off. 
AllowBreak = 1 will break words

TF_Wrap("File.txt",60)            ; wrap at col 60

TF_WhiteSpace(Text, RemoveLeading = 1, RemoveTrailing = 1, StartLine = 1, EndLine = 0)

Purpose: Remove leading and/or trailing whitespace
Parameters: Text, RemoveLeading (0 or 1), RemoveTrailing (0 or 1), StartLine, EndLine

RemoveLeading = 0  Do not remove leading white space of lines
RemoveLeading = 1  Remove leading white space of lines
RemoveTrailing = 0 Do not remove trailing white space of lines
RemoveTrailing = 1 Remove trailing white space of lines

TF_WhiteSpace("File.txt", 1, 0, "5-10") ; remove leading and keep trailing whitespace in lines 5 to 10

TF_Substract(File1, File2, PartialMatch = 0)

Purpose: Delete lines from file1 in file2 (using StringReplace)
Parameters: File1, File2, PartialMatch (0 or 1)

File2: you can use !file2 to overwrite file2 otherwise output is file2_copy.txt
PartialMatch = 0 lines from file1 must appear as is, case insensitive 
PartialMatch = 1 allow for paRTIal match of line
PartialMatch = 2 allow for paRTIal match of line, but remove entire line (uses RegEx, new as of v3.4)

TF_RangeReplace(Text, SearchTextBegin, SearchTextEnd, ReplaceText = "", CaseSensitive = "False", KeepBegin = 0, KeepEnd = 0)

Purpose: A Range Replacement allows you to perform a replacement on text whose beginning and ending remains the same, but whose middle contents might change.
Parameters: SearchTextBegin, SearchTextEnd, ReplaceText, CaseSensitive (True or False), KeepBegin (0 or 1), KeepEnd (0 or 1)

Note: Similar to "BK Replace EM" Range Replacement. It is basically an easier shortcut for a more complex RegExp.

KeepBegin = 0 Remove SearchTextBegin from file
KeepBegin = 1 Do not remove SearchTextBegin from file (saves you the trouble of putting it back in via ReplaceText)
KeepEnd = 0 Remove SearchTextEnd from file
KeepEnd = 1 Do not remove SearchTextEnd from file (saves you the trouble of putting it back in via ReplaceText)

SearchTextBegin = "" e.g. empty means from START of File *1
SearchTextEnd = "" e.g. empty means until END of File *1

*1 if you pass on empty strings to SearchTextBegin and SearchTextEnd the entire file will replaced with the replacement text. SearchTextBegin/End can be on the same line. Remember this LIB mainly operates on a line by line basis. This Function only operates on the FIRST SearchTextBegin to the FIRST SearchTextEnd it finds.

Range=[insert this`ntext for the`nrange replace Text`ntest function]
TF_RangeReplace("File.txt", "Create hotkeys for keyboard", "into an EXE file", Range)

TF_MakeFile(Text, Lines = 1, Columns = 1, Fill = " ")

Purpose: Create file of X lines and Y columns, fill with space or other character(s)
Parameters: TextFile (new file to be created), Lines, Columns, Fill

Sometimes you need an "empty" file before you can start adding line numbers or putting data into a file.

TF_Tab2Spaces(Text, TabStop = 4, StartLine = 1, EndLine = 0)

Purpose: Convert tabs to spaces, shorthand for TF_ReplaceInLines
Parameters: Text, TabStop, StartLine, EndLine
Credits: incl. ideas from infogulch

TabStop = number of spaces to replace a TAB with, so 4 means a TAB will be replaced by 4 spaces

TF_Spaces2Tab(Text, TabStop = 4, StartLine = 1, EndLine = 0)

Purpose: Convert tabs to spaces, shorthand for TF_ReplaceInLines
Parameters: Text, TabStop, StartLine, EndLine
Credits: incl. ideas from infogulch

TabStop = number of spaces to replace with a TAB, so 4 means 4 spaces will be replaced by a TAB

TF_Sort(Text, SortOptions = "", StartLine = 1, EndLine = 0)

Purpose: Sort (section of) TextFile
Parameters: Text, SortOptions, StartLine, EndLine

SortOptions: use the SORT options http://www.autohotkey.com/docs/commands/Sort.htm

Note: StartLine can not have multiple sections, increments or multiple lines in this case. When dealing with variables/strings instead of text files the native AHK Sort command is of course more useful.

TF_Join(Text, StartLine = 1, EndLine = 0, SmartJoin = 0, Char = 0)

Purpose: Join lines (section of)
Parameters: Text, StartLine, EndLine, SmartJoin, Char

SmartJoin: Detect if CHAR(s) is/are already present at the end of the line before joining the next, this to prevent unnecessary double spaces for example.
Char: character(s) to use between new lines, defaults to a space. To use nothing use ""

Note: StartLine can not use increments. It can have multiple sections.

TF_Save(Text, FileName, OverWrite = 1)

Purpose: To save a variable to a file
Parameters: Text, FileName, OverWrite (O or 1)

OverWrite = 0 create filename_copy.ext if filename.ext exists
OverWrite = 1 will overwrite filename.ext if filename.ext exists (default)

History

History v3.4, 30 October 2010 - UNRELEASED at the time

New function: TF_Join: joining lines.
Added: A_ThisFunc to helper function TF_MatchList, this may help with error messages such as: Invalid StartLine parameter (non numerical character) - Function used: A_ThisFunc
Added/Change: New option for StartLine, Negative startline operate on last X lines was already the case for TF_removelines, applies to all TF functions
Added: TF_Substract: Partialmatch = 2 now deletes entire line on partial match, see TF DOC for more info.
Added: TF_ColGet and TF_ColPut now accept negative StartColumn parameter. Can be used to Get or Put text in the xth "column" from the right, see description above - TF_ColCut does not support this as you can use TF_TrimRight.
Added: TF_ColGet, TF_ColPut and TF_ColCut can now process multiple columns in one go. Format: CSV or incremental (DO not use sections for TF_ColPut due to unexpected end results)
Minor improvements to the documentation
removed TF_FindLines from lib (was already deprecated)

History v3.3, 16 April 2010

Fixed: If you used variables with any of the replacement functions or tried to remove empty lines from a VARIABLE in the following format: variable:=TF_..(variable,"search","replace") and the searchtext was NOT present in the variable it returned an empty variable (e.g. deleted the contents of "variable"). It affected TF_ReplaceInLines, TF_Replace, TF_RegExReplaceInLines, TF_RegExReplace, TF_RemoveBlankLines, TF_RangeReplace. The built-in check only worked correctly for FILES and with the introduction of variables in TF 3 this didn't surface during testing.
Fixed: documentation error for TF_Merge (separator and filename where swapped) and added examples on how to use TF_Merge in a Loop and with FileSelectFile

History v3.2, 20 February 2010

Fixed: TF_AlignRight: due to a bug it didn't work as it should have, it prepended the number of spaces rather than aligning the text at the specified width.
Changed: TF_GetData (helper function) should now avoid unnecessary IfNotExist for files, for scripts with many loops in combination with variables it should improve the speed slightly
Changed: TF_SplitFileByLines: New options for SplitAt, now three methods available, see notes at function description
Changed: TF_RemoveLines: New option for StartLine, if negative value is used it will remove the last X lines from file, see notes at function description

History v3.1, 09 December 2009

Changed: Rewrite of TF_Find. Can now return multiple lines (like TF_Findlines used to), not only linenumbers but the entire line (text) of found lines so it can be used as a basic grep. Now uses RegExp, see compatibility notes.
Deprecated: TF_FindLines (see change TF_Find). Kept in for backwards compatibility
Updated and expanded documentation with examples and a hopefully "better" introduction highlighting some basic concepts

History v3.0, 27 November 2009

Complete overhaul of library, now accepts files & variables for input and output:

Changed: New parameter for TF_Readlines & TF_Tail: trailing new line now optional
Added: TF_Save, shorthand for filedelete+fileappend
Added: TF_GetData, helper function to determine if a file or a variable is passed on to function
Added: TF() To read a text file in global var, t by default - Credits various ...
Added: TF_ReturnOutput has replaced: Overwrite, MakeCopy and the newly developed ReturnStr
Changed: Complete rewrite of TF_Tail, new options - borrowed bits from Tuncay (Thanks!)
Changed: MakeMatchList: Removed TF_Countlines (one less fileread), Pass on "String" and not a TextFile
Fix: TF_ConCat & TF_MakeFile didn't write output file - same bug as splitby* earlier.

History v2.5 fix, 01 November 2009

Unreleased, but available on request :-) Note that in 2.4a TF_MakeFile and TF_ConCat do not produce output files, easy to fix if you are determined to use an older version of TF.

History v2.4a fix, 10 August 2009:

The fix for TF_SplitFileBy* functions of 2.3b wasn't complete, now it should work correctly (it didn't write the last file)

History v2.4 update, 06 August 2009:

Added: TF_Sort

History v2.3b update, 03 August 2009:

Fixed: No output problem for TF_SplitFileBy* functions, "bug" introduced by 2.3a "Built in Check if TextFile actually exists"
TF_RemoveBlankLines check if file has empty empty lines to start with, if not return and do nothing (does not create file_copy identical to file)
TF_RangeReplace same fix as 2.3a

History v2.3a update/bugfix (29 July 2009, HugoV, ribbet.1, Murp|e)

Fixed/Changed:

New features in TF_LineNumber: Restart + Choice of leading/padding character. Thanks for the idea ribbet.1, http://www.autohotkey.com/forum/viewtopic.php?p=284687#284687
Built in Check if TextFile actually exists, to prevent creation of empty file(s). Thanks for the idea Murp|e, http://www.autohotkey.com/forum/viewtopic.php?p=284649#284649
TF_Replace: if SearchText wasn't present in TextFile the function never returned (stuck in endless loop)
TF_ReplaceInLines: if SearchText wasn't present in TextFile simple return an do nothing (faster, does not create file_copy)
TF_RegExReplaceInLines: if NeedleRegEx wasn't present in TextFile simple return an do nothing (faster, does not create file_copy)

History v2.3 (28 July 2009)

Added: TF_Tab2Spaces - Note: Thanks to infogulch
Added: TF_Spaces2Tab - ditto
Added: TF_RangeReplace
Added: TF_MakeFile

Also thanks to Murp|e for pointing out some errors in the documentation (TF_COL* functions).

History v2.2 (10 July 2009)

Added: TF_Substract
Added: TF_WhiteSpace
Added: TF_Wrap

History v2.1

Added: TF_Prepend
Added: TF_Append

History v2.0 (Overhaul by HugoV using TXT lib as starting point)

Renamed: TF_TotalLines to TF_CountLines (seemed more logical)
Removed: TF_GetCSV, TF_SetCSV to keep the focus on TXT files, CSV requires a different library IMHO (HugoV)
Introduced: MakeMatchList and adjusted TF_ReadLines, TF_RemoveLines, TF_InsertPrefix, TF_InsertSuffix, TF_TrimLeft, TF_TrimRight, TF_ColGet, TF_ColPut, TF_ColCut accordingly (HugoV)
- Make a MatchList which is used in various functions (listed above)
- Using a MatchList gives greater flexibility so you can process multiple sections of lines in one go avoiding repetitive fileread/append actions
- You can pass on multiple lines (sections) If you quote the StartLine parameter. Examples of StartLine, Endline: 5 ; start from line 5 to the end (StartLine) 5, 15 ; lines 5 to 15 (StartLine, EndLine) "5,13,45,67", 135) ; lines 5,13,45,67, ignore 135 ("StartLine", Endline - EndLine value is ignored) "5-13,45-51", 135) ; lines 5 to 13, 45 to 51 ignore 135 ("StartLine", Endline - EndLine value is ignored)
- Incremental processing of lines, example startline, endline - endline will not be ignored "2+3", 150 ; start with line 2, increment 3 up-to line 150 so 2,5,8,11,14 etc "5+15" ; start with line 5, increment 15 until end of file so 5,20,35 etc
Introduced: OverWrite & MakeCopy (Heresy / HugoV)
- No longer necessary to pass on full path, e.g. "file.txt" works OK now (HugoV)
- Reduces size of library by removing repeating code sections for saving Output (Heresy)
- Backup files: If a subdirectory "backup" is present in the directory of TextFile a backup is made before overwriting the original file (both for file.txt and file_copy.txt) with the BAK extension (HugoV)
- Includes fix to remove trailing `n added by most functions (HugoV)
Introduced: SetWidth/space by SKAN for TF_Align* Functions http://www.autohotkey.com/forum/viewtopic.php?p=45880#45880
Added: TF_ReplaceInLines. Similar to TF_Replace
Added: TF_RegExReplaceInLines. Similar to TF_RegExReplace
Added: TF_SplitFileByLines (HugoV)
Added: TF_SplitFileByText (HugoV)
Added: TF_Merge (HugoV)
Added: TF_Find (HugoV, based on example by olegbl)
Added: TF_FindLines (HugoV, based on example by olegbl)
Change: TF_TrimLeft. 1 - it now processes all lines of the TextFile. 2 - you can now omit EndLine e.g. will process until end of file (HugoV)
Change: TF_TrimRight. 1 - it now processes all lines of the TextFile. 2 - you can now omit EndLine e.g. will process until end of file (HugoV)
Change: TF_AlignCenter, TF_AlignLeft, TF_AlignRight. Now use SetWidth/space by SKAN, more reliable also adds spaces to RIGHT side of line
Change: TF_AlignLeft. 1 - you can now use StartLine, EndLine parameters. 2 - StartLine and/or EndLine can be omitted (HugoV)
Change: TF_AlignRight. 1 - you can now use StartLine, EndLine parameters. 2 - StartLine and/or EndLine can be omitted (HugoV)
Change: TF_ReadLines you can now omit EndLine e.g. will read until end of file (HugoV)
Change: TF_RemoveBlankLines you can now use StartLine, EndLine parameters. EndLine can be omitted (HugoV)
Change: TF_InsertLine modified to accommodate line sections and incremental lines.
Change: TF_ReplaceLine modified to accommodate line sections and incremental lines.
Change: TF_RegExReplace no longer returns "Counts" (HugoV)
Change: TF_RemoveDuplicateLines 1 - no longer returns "Removed". 2 - can use start/end line, 3 - no longer uses sort but keeps original order intact, 4 - to methods to check consecutive lines or unique files in file (HugoV)
Change: TF_ColGet, TF_ColPut, TF_ColCut. 1 - order of parameters changed, 2 - now accept Startline, EndLine parameters. (HugoV)
Fixed: TF_RemoveDuplicateLines always produced empty Output file (HugoV)

History prior 2.0 - TXT library (Heresy, HugoV)

2008-06-18 : added TF_AlignLeft(), TF_AlignCenter(), TF_AlignRight() (Heresy)
2008-06-18 : added TF_Tail(), TF_ReverseLines() (HugoV)
2008-06-17 : added TF_GetCSV(), TF_SetCSV() (Heresy)
2008-06-17 : added TF_LineNumber(), TF_Concat(), TF_ColGet(), TF_ColPut(), TF_ColCut() (HugoV)
2008-06-16 : added TF_RegExReplace(), TF_RemoveLines(), TF_TrimLeft(), TF_TrimRight() (Heresy)
2008-06-15 : added TF_TotalLines(), TF_ReadLines(), TF_RemoveDuplicateLines() (Heresy)

Credits

This library is based on the Library for Text file manipulation started by Heresy. I have contributed a number of functions to that library, but that version also has some "bugs" which are hopefully resolved in this TF library.

Thanks to: Heresy, SKAN! (countlines, setwidth, spaces), Olegbl, (suggestion for find*), infogulch (suggestion for tab <-> spaces), Murp|e (suggestions for documentation, check if file exists), ribbet.1 (New features in TF_LineNumber), Tuncay (help with TF() and borrowed ideas for TAIL).

Name		Name	Last commit message	Last commit date
Latest commit History 3 Commits
license.txt		license.txt
readme.md		readme.md
tf.ahk		tf.ahk

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

TF: Textfile & String Library for AutoHotkey [lib] - v3.4

Introduction

Functions

How to "install" / use

Quick intro to Parameters

(StartLine, Endline) Syntax

Textfile and the ! Prefix

Functions and Examples

History

Credits

About

Releases

Packages

License

vondravl/TF

Folders and files

Latest commit

History

Repository files navigation

TF: Textfile & String Library for AutoHotkey [lib] - v3.4

Introduction

Functions

How to "install" / use

Quick intro to Parameters

(StartLine, Endline) Syntax

Textfile and the ! Prefix

Functions and Examples

History

Credits

About

Resources

License

Stars

Watchers

Forks

Releases

Packages 0

Packages