Page 1 of 2
C# Compatible File System
Posted: Fri May 16, 2014 12:38 am
by PGilbert
I would like to write from APL a simple file containing text and numbers that could be read also by C#. The obvious choice would be to convert the APL variable to XML or JSON and write the result as a text file. JGoff suggested a way to fill a HashTable with an Apl NameSpace but it is not obvious that C# will be able to decipher easily the HashTable into .Net variables.
Is someone had this problem to resolve before ? In the affirmative what was the solution retained.
Re: C# Compatible File System
Posted: Fri May 16, 2014 12:06 pm
by PGilbert
So far the best solution that is working for us is to use an HashTable like JGoff suggested but with a JSON character representation of the APL data. The only problem is that we store large simple numeric array (1000 x 21) and the JSON parser is taking too much time to parse it back in APL. One solution is to test if any '"' character is present in the JSON string and if not present this is a simple numeric array (or particular case) and it can be parse to APL with {(v[1]↑1↓v)⍴(1+v[1])↓v←⍎(','⎕R' ')1↓¯1↓⍵} wich is very fast for that particular case.
Re: C# Compatible File System
Posted: Fri May 16, 2014 2:42 pm
by jGoff
If I were looking at this again, the first thing I would to is take off my APL hat and not pass everything through the same code, i.e. not code and decode everything into JSON or anything else. I don't know the profile of your data, but if most of it is .Net friendly, as most data usually is, then that will increase your efficiency with your having to do much work.
Re: C# Compatible File System
Posted: Fri May 16, 2014 9:48 pm
by PGilbert
Hello Jeff, I am not sure that I understand you. But you are hinting me that the program should write a .Net object into the HashTable (of type String, Array or DataTable for exemple) so that when it will be deserialize in C# it will return the same object that will be recognize and usable. If I put an APL character vector (or APL matrix) in the HashTable it will be meaningless in C# once deserialize. So ideally I should write a .Net object in the HashTable that represent the APL vector or matrix and that way I don't need to go through JSON. And even the JSON APL character vector should be written has a .Net object for the same reason.
Is this what you are explaining to me ?
Re: C# Compatible File System
Posted: Sun May 18, 2014 12:37 pm
by DanB|Dyalog
Using
is a good idea but remember that ⎕VFI is even faster than ⍎.
Even better, Dyalog has dyadic ⎕VFI where you can specify what delimiter to use on the left.
This means you can drop ⎕R and do
instead and shave off a few more micro seconds.
Re: C# Compatible File System
Posted: Sun May 18, 2014 1:37 pm
by PGilbert
Thanks Daniel, for a large file the time required went from 138 ms to 13 ms. So ⎕VFI is really faster like you said. Looks like the final idea would be to use an HashTable but with a JSON representation of the APL data. That way it could be parse back in C# or other language. I am not an expert in JSON but here is what we will be using in our particular case for simple array and character vector combine with an HashTable as described in Jeff post:
Code: Select all
SimpleNumericArraytoJSON←{⎕PP←10 ⋄ 1⌽'][',(('¯'⎕R'-')(' '⎕R',')⍕(⍴⍴⍵),(⍴⍵),,⍵)}
JSONtoSimpleNumericArray←{(js[1]↑1↓js)⍴(1+js[1])↓js←2⊃','⎕VFI('-'⎕R'¯')1↓¯1↓⍵}
SimpleAplTextToJSON←{1⌽'][',((' '⎕R',')⍕(⍴⍴⍵),(⍴⍵)),',',1⌽'""',⍵}
JSONtoSimpleAplText←{(¯1↓1↓2⊃','⎕VFI 1↓(¯1+⍵⍳'"')↑⍵)⍴¯2↓(⍵⍳'"')↓⍵}
Any idea to improve the speed is welcome again.
Re: C# Compatible File System
Posted: Sun May 18, 2014 11:48 pm
by jGoff
[Pierre]
SimpleNumericArraytoJSON←{⎕PP←10 ⋄ 1⌽'][',(('¯'⎕R'-')(' '⎕R',')⍕(⍴⍴⍵),(⍴⍵),,⍵)}
JSONtoSimpleNumericArray←{(js[1]↑1↓js)⍴(1+js[1])↓js←2⊃','⎕VFI('-'⎕R'¯')1↓¯1↓⍵}
[Jim]
Using the "Dyalog .Net Interface" to pass a simple APL numeric array to C#, will get you a C# array of the same type, rank, dimensions, and item values. Why the extra JSON processing?
Re: C# Compatible File System
Posted: Mon May 19, 2014 1:52 am
by PGilbert
To Jeff: Because I know nothing about C# !!!! I was under the impression that the HashTable would contain the Dyalog serialization of the internal representation of the APL array or character vector and the programmer in C# would need to decypher that. How come the HashTable get the 'Type' correct from Dyalog and use it to save it to the HashTable ?
Re: C# Compatible File System
Posted: Mon May 19, 2014 9:09 am
by DanB|Dyalog
Pierre,
I don't know if this may help you but there a exist JSON parser in SALT.
For example, if you do
then you will have access to some tools to deal with JSON
Code: Select all
⎕←rep←#.JSONdb.fromAPL 2 3 4⍴⍳9
[3,2,3,4,1,2,3,4,5,6,7,8,9,1,2,3,4,5,6,7,8,9,1,2,3,4,5,6]
#.JSONdb.toAPL rep
1 2 3 4
5 6 7 8
9 1 2 3
4 5 6 7
8 9 1 2
3 4 5 6
note that <fromAPL> encodes the rank and shape with the data.
Re: C# Compatible File System
Posted: Mon May 19, 2014 9:36 am
by JohnD|Dyalog
Hi Pierre,
If you create a HashTable in APL, and add items to it from APL, then they will be put in the HashTable in an internal .Net representation that the C# code at the other end will be able to read. The data will not be written in APL's internal format.
Your process would be:
In APL:
1) Create the HashTable
2) Add all your name/value pairs from APL
3) Serialize your HashTable (via .Net, to XML, or JSON, or binary, whatever you prefer), to a file
In C#
1) Deserialize the file to a HashTable
2) Read all the elements from the HashTable into C#.
Of course, this sort of assumes that your data is in name value pairs. If not I guess you could just add a big nested array to the HashTable, and serialize/seserialize it in the same way.
Best Regards,
John Daintree.