Tradestation and Multicharts Fileappend command is too slow for many operations. Here i would like to introduce a solution
This library is not dedicated for Tradestation. It will work with every other platform such as Multicharts, Ninjatrader, C#. Our demonstration here is just for demo.
Lets imagine you want to export OHLC Data as CSV to be able to analyse from an external tool. The problem is that Fileappend will be very slow operating, this is probably because Tradestation has only single command (its easy to handle) to Append and it probably does open and close by every single line and this requires time.
If we wanted export all Bar OHLC Data as external .csv file we need a code like seen:
// code snipplet OHLC="ohlc=O="+Numtostr(o,6)+";" +"H="+Numtostr(h,6)+";" +"L="+Numtostr(l,6)+";" +"C="+Numtostr(c,6)+";" +"DATE="+Numtostr(Date,0)+";" +"TIME="+Numtostr(TIME,0)+";" +"VOLUME="+Numtostr(Volume,1)+";"+Newline; Fileappend(FN,OHLC);
Exporting 1 Year of M1 Data contains around 726000 Bars in Tradestation (EURUSD), to export all these bars Fileappend() needs 65 seconds to complete the operation (at least on my very fast computer with Flash/SSD Harddisk). If you need to export 10 years of data it would require over 12 minutes to complete, not very efficient.
I have developed a library called fast_appender.dll, this library is allowed to accept lines to append, it does not open, close by every single append operation. It opens the file and keeps appending until you say to close.
The import is like seen:
external: "fast_appender.dll", string, "faVersionA"; external: "fast_appender.dll", int, "faInitA",LPSTR,int; external: "fast_appender.dll", int, "faAppendA",int,LPSTR; external: "fast_appender.dll", int, "faClose",int;
As you see FastAppender is over 5 times faster in appending to text files. 5 times speed improvement is not something you should underestimate, there is huge difference between 5 seconds and 25 seconds per chart and symbol and exported file.
int faInitA(string Filename,bool Append)
This is first function you need to call and call it only once (once begin….end block) at start. First parameter is name of file, second parameter accepts either 0 (false) or 1 (true), if you call it with 1, it will Append to existing file, if you call it with 0, it will Overwrite file and start Appending from first byte. This function returns a handle as int, you need to remember this handle, it will be reused in other commands
int faAppendA(int Handle,string LineToAppend)
This function will append LineToAppend to previously opened File, the Handle is taken from faInitA.
int faClose(int Handle)
This closes the handle, you will not need this function in Multicharts / Tradestation, you can need this function if you need to call from other platforms. Once you close the handle, you also close the file and you cannot Append anymore. If you are sure you will not append anymore, then you can close it.
Returns Version information of library.
This example exports OHLC Data of all bars as .csv file
Inputs: ExportDirectory("c:\DATA1\"); Vars: OHLC(""),intrabarpersist FN(""),BInterval(0),intrabarpersist fa(0); // Import fast_appender external: "fast_appender.dll", string, "faVersionA"; external: "fast_appender.dll", int, "faInitA",LPSTR,int; external: "fast_appender.dll", int, "faAppendA",int,LPSTR; external: "fast_appender.dll", int, "faClose",int; Once Begin if BarType=1 then BInterval=BarInterVal; if BarType=2 then BInterval=24*60; if BarType=3 then BInterval=24*60*7; if BarType=4 then BInterval=24*60*30; If BarType=12 then BInterval=(Range/1 Point); FN=ExportDirectory+getsymbolname+"-"+Numtostr(BInterval,0)+".csv"; // open File Handle and remember Handle in fa variable fa=faInitA(FN,1); end; OHLC="ohlc=O="+Numtostr(o,6)+";" +"H="+Numtostr(h,6)+";" +"L="+Numtostr(l,6)+";" +"C="+Numtostr(c,6)+";" +"DATE="+Numtostr(Date,0)+";" +"TIME="+Numtostr(TIME,0)+";" +"VOLUME="+Numtostr(Volume,1)+";"+Newline; if (fa>0) then faAppendA(fa,OHLC);
You can download it here
FastAppender library is working with threads, you submit the data which needs to be appended and it will do the actual work in background to ensure speed. This library is over 5x faster than native command. It replaces native Fileappend command completely, you can use it for all your Append, Export operations.Have fun using it