Programmer's manual

 
A small warning here. I will add sometimes some suggestions to Doc Scilab (well to the Scilab Team). And it will be always in a funny way and sometimes in laughing a little at them. This is just an effect of my bad caracter. In any case, I express my thanks and my support to the SciLab team. They gave me a language that allowed me to write this application in a fraction (1/20 at least) of the time that would have been required in C or C++ without any library. And they put it free which is even better. I will never thanks them enough and ask them enough to continue to improve their language. The remarks, I will do, just have a goal : show the few shortages I found in scilab and asked them to be suppressed in future releases.
 
Informations hereunder are exact, however the different structures have regular additions (a comment for the signals, values for choosing the vertical scale, sampling period, new scilab signals, ...). The ultimate reference is the code which is supposed to be clear (nearly everywhere).

1. Welcome

 
Well there is a risk that you suffer. Basically, this program is simple because it is just a graphical interface to scilab commands. However, the details are not so simple (well not so hard also).
 

2. Data structures

 
SigLab is a tango between 2 partners : a signal set and a screen. The music is what you ask. And the ground is in fact a memory buffer. All signals are stored and loading or processing is only made when the given values are not directly available.
 
So the most important thing to understand is the different data structures that are used. There are 4 :

• the file list,
• the matrix list,
• the graphic order,
• the graphic list.

 
In the very first version of SigLab, these structures were sent from one procedure to the other classically. Thanks to Scilab 2.5, these structures are now global. The only thing sent (when required) is the index(es) of signal(s) to process. It speeded up much SigLab.
 

2.1. The file list

The file list will contain all informations needed to "load" a file. The term "load" has different meanings in function of the signal type. It may be either a normal file on a physical support, a worked out signal , or a "matrix" signal (a "file" which is fully in memory) or a scilab signal (a signal linked to a scilab variable).
 
A file list is a structure with the following elements : name fid line file_size type element_size matrix first_element_mf nber_of_element_mf.
 
The element functions are the following :
 

• name : name of a file for a physical file. For a worked out file, it is the formula used for working out the displayed elements. For a matrix file, nothing particular was defined.
• fid : For a physical file, the file id (the pointer on the current element of the physical file) if the file was previously opened else -1. If the file is a worked out file (coming from a calculation) then it is -2. If the file is a matrix file then it is -3.
• line : line nber on which we stopped reading
• type : the file type (asc ascii; calc - formula; d,f,c,l,ul,... : binary files;matrix : matrix file)
• file_size : the estimated nber of elements (in fact lines) in the file
• matrix : if all values of a "file" are stored in memory, then we have a matrix file. The values are stored as a vector in the matrix element of the file list. It is just a matrix in which there are all values known for a given signal.
• first_element_mf : the sample nber of the first element of the matrix file. Elements before will be considered as being 0.
• nber_of_element_mf : the nber of element of the matrix file. Elements after first_element_mf + nber_of_element_mf - 1 will be considered as 0.

 
The basic creation function is in the file "create_file_list.sci".
 
For each signal that is opened, a file list is created with these informations. The file lists of all "opened" signals are regrouped in a list. In the main program, this list is loFL (list of File List). This mecanism is used for the 3 main structures (file list, matrix list and graphic orders).  If anybody wishes to directly access a particular element, it is just required to ask for loFL(1)('matrix') to get the matrix element of a signal. It may be an empty matrix.
 
3 things would be a good idea : to add a "legend" element in order to have a possibility to add a textual legend to a signal independently of its exact name. The display of the legend would be managed from the graphical manager dialog with a checkbox. The second thing would be to maintain a dependency list and an inverse dependency list (which signals depend on me), in order to warn when a signal is deleted that it is necessary to another signal.
 
As you understood, the file list does not contain really the values that are displayed (except for matrix files). It just contains informations on how to find the values that will be displayed.
 

2.2. The matrix list

 
We now need to store in memory the values that will be displayed. This is done in the matrix list structure. This matrix list structure is especially used to maintain a buffer for each signal will values for this signal. This buffer is in fact a window of the signal. This buffered window is different from the display window. The goal is to load or work out signals only when required (when the values where not previously read) and then to store them as long as possible. It means that "continuous" motions will be fast while jumping to start and then end of file will be slow. You are warned. The idea here is that siglab was planed to work with files with million of points and that somebody will concentrate and study a portion of a file. Step after step, the window of samples will be fully in memory and it will be quick to work on them. Matrix file is in fact the famous "ground" from the tango.
 
The elements of the structure are the following : matrix name first_element nber_of_element is_complex.
 
The element functions is the following :

• matrix : the buffered values. And this is for all type of signals. It is in these values that the displayed samples are taken.
• name : the name of the signal. Strictly the same value as what is in file list. May be suppressed one day. However, it allows to just use the matrix list without sending also the file list.
• first_element : the index of the first element (the abscisse in fact).
• nber_of_element : the length of the buffered matrix. Strictly redundant with length(loM(x)('matrix').
• is_complex : a late addition due to some dissymetry in the way complex and real values are handled in scilab (try to plot a complex signal to see what I mean : it gives an error). Then, I added a tag that says if a signal is complex or not. Once it is worked out, there is no need to decide again. Because it is a slow process to decide if a signal is complex or not. There is here a missing function since SciLab internally knows perfectly if a signal is complex or real. It changes the way the signal is stored ( Doc Scilab are you here ? got it ?).

 
You will find basic functions that manipulate a matrix list in the file matrix_list.sci (surprising, isn't it ?). Especially, there are functions to glue 2 matrix lists in a single one. The buffers are appended, the sizes are added. Another functions will cut elements from a matrix list buffer.
 
As for the file list, there is a matrix list for each signal which is processed. And all matrix list are maintained in a list of matrix list the name of which is loM in the main program.
 

2.3. The graphic order

 
We now need to know which part of the buffer we wish to show. This is done with the graphic order structure. This structure is going to say how a signal will be displayed. The elements of the structure are the following : first_element nber_of_element frame_nber style display grid.
 

• first_element : the index of the first point to display,
• nber_of_element : the nber of points to display,
• frame_nber : the frame nber where it will be displayed. This is the place on the graphic window where this graphic_list should be shown,
• style :  the style of the graphic. In fact, it is just the color number. It should be possible to change it from the graphic manager. I should also investigate how to put crosses, diamonds, ... and not only continuous line,
• flag : display(1) / do not display(0).
• grid : off (0) / on (1).

 
So for displaying in a certain way a curve, it is just required to put in the corresponding graphic order how, where and if you want to see it. There are a bunch of functions that will take that and do all the processing to bring it on screen. (more on that later). When the graphic manager window is shown, nearly all these values for all available signals are sent to the window and displayed in a manageable way. When the graphic manager window is closed, these values are sent back and updated in the graphic orders.
 
For each curve processed, there is a graphic order, and they are grouped in a list. The name of the list in the main program is loGO (list of Graphic Order).
 

2.4. The graphic list

 
The graphic list is in fact something that you should never have to work with. It is just the final structure containing what will be shown. There will be a graphic list for each panel in the graphic window. If many curves are on a given panel, then a matrix is constructed with one column for each curve.
 
The elements of the graphic list structure are the following : x y style name index_list grid.
 

• x is the abscisse vector,
• y is a matrix with the points displayed. There is one row for each curve (except for complex signals). A complex signal will be displayed with 2 curves the real part and the imaginary part.
• style : the style of the curves. It is a vector of all styles from the curves in the panel.
• name : a vector with all names of the different curves. The different names are separated by @, and I add Si after the name of the curve.
• index_list : the list of index of the signals that are the curves. I think that I do not use it today,
•   grid : if equal to 1 then a grid is displayed. If any curves on the panel requires the grid, I display it.

 
Procedures related to graphic_list are in the file graphic_list.sci.
 

3. The basic loop

 
There is a way in which the list of structures previously defined are always processed in order to bring a display. I am going to give a first rough idea of it now.
 
First, you may put 1 after the DEBUG variable that is at the start of siglab.sci. It will slightly slows down the display, but it will show you the function calling tree clearly. The indentation corresponds to the calling level. This is implemented with the debug_function and leaving_debug functions.
 
Let's suppose that you have created a list of graphic order. You want now to execute this graphic order. If you look at all functions you will see something like :
 
        // Next Synchronize everything
        synchronize_lo(maximum_window_size);
        synchronize_lo_calc(maximum_window_size);
 
        // Create the graphic_list from the matrix list and the graphic_order list
        graphic_list=ml2gl();
 
        // Now displaying the result
        display_graphic_list(graphic_list);
 
The first step for displaying is to synchronise the internal buffer with the graphic order (synchronize_lo). This corresponds to do what is necessary to have in memory (in the list of matrix list) at least all samples that are necessary for the display. Once this loading is done for all physical files and matrix files, the "loading" (the calculation) of the worked out signals is processed. This is the synchronize_lo_calc function.
 
In both case, you see that the maximum_window_size is given in parameter. If what is asked is to big it will not be processed. When all samples are in memory, it is time to really prepare what will be displayed. Especially, if they are many curves on the same graphic plane, or if they are signals not to be displayed, it will be taken into account. This is done by the ml2gl function (matrix list to graphic list function). As you see, the loFL (list of file list) is no more required here as everything is in memory.
 
The function ml2gl goes through the list of matrix list and the corresponding graphic order and construct the list of the graphic data for the different panels in the graphic window. In fact, it is able to fully change the order of everything. If you look inside this function (file graphic_list.sci), you will see that it calls a "create_curve" function. This function extracts from the matrix list of a curve the vector of samples that will be displayed and all associated informations (name, style, grid, ...). In fact, what is important here, is that the type of the file (physical, worked out, matrix file) does not have any importance.
 
Finally, once the list of graphic list is built, it is displayed thanks to the display_graphic_list function (file graphic_list). If you look at the function you will see that it is trivial, and that there is a lot of potential improvements (change the background color of graphics, prepare a b/w version for printing, change the polices used, allow logarithmic display, ...).
 
If you understood this basic loop, you are ready to write new things in Scilab. You certainly see that you should nearly never interact with this loop. When adding functions to SigLab, it will be done certainly before this basic loop.
 

4. Details on physical files, worked ou files and matrix files

 
The key to understand the physical file management is the load_file function (load_file.sci file). This function will load nber_of_element samples of the argument file list, from the sample the index of which is first_element. This function is "intelligent" and will decide what to do in function of the file type. If it is a physical file, it will open the file if necessary, determines its type (binary or ascii, complex or not, if binary, long, short, float, double, char, ...). It is able to jump from one part of a file to another depending of the file type (obviously if it is a binary file it is quicker because the length of each element is known).
 
Then the load_file function does the following :

• verify that a file is opened,
• jump at the location in the file where is the first character to read,
• calc the read_(ascii,bin,complex,complex_bin,calc,matrix) function for reading the elements,
• put 0 where it was not possible to read the samples,
• send back a matrix_list with all read elements.

 
I am not going to enter in the details of reading ascii, bin, complex, complex_bin files because it is just file manipulation and you just have to read the code to understand it. It would be a good idea to add the wave format file support and it could be done in customizing these functions without much difficulties.
 

4.1. Worked out files

 
So in fact, worked out files are really worked out in the read_calc function.
 
If you look at the name of a worked out file (for example S1 + S2) you will see something like :
loM(1)('matrix')(start_indice:last_indice) + loM(2)('matrix')(start_indice:last_indice)
 
So to "load" the corresponding points, this string is evaluated with an evstr instruction within the read_calc function. You certainly understand now that based on the functions available within SciLab, it is fairly trivial to have sinus, cosinus, +, ... buttons. The point is just to create the string that will be adapted to scilab.
 
We will look in more details, how the graphical interface is organised and how the string is created in a next part of this documentation.
 

4.2. Matrix files

 
Matrix files are very simple. In the corresponding File List, a matrix exists with all values of the signal (element 'matrix' of the file list). The abscisse of the first point of this matrix, and the number of points are also stored.
Whenever a point is required, the sample is searched in the 'matrix'. If it is not there 0 is returned. You may see that in the read_matrix function of the load_file.sci file.
A matrix file is created with the function :
new_signal_from_matrix_file
It is heavily used in the noise generation, or for the fft, ifft, spectd, ... processing.
 

4.3. Files

 
Classical files are trivially processed. You will see the boring reading of each file type in load_file.sci (ascii, double, char, long int, short int, unsigned ..., complex ascii, ...). Complex signals are stored with real part of a number first, imaginary second.
The big work is to succeed to ask for only the samples that are not in memory. This is done before the call of load_file.
 

5. Tcl/Tk interface

 
I started to developp with SpecTcl (the main calculator) but I switched just 2 months ago to tkBuilder. Believe me, tkBuilder is good, no, excellent, no, marvellous, no, it is THE TOOL for people programming complex interfaces and that want to have full control on what they do. Get it.
 
I joined all tkb files that I have. However, they are not always synchronised with the final files. So be cautious. My development cycle is :

• design the interface with tkBuilder and play with it,
• generation of the tcl (file toto_win.tcl),
• change of the tcl to integrate it in SigLab. I previously did that manually, but succeeded to write some nice sed function that does that. Unfortunately, I did not fully resynchronised (yet) the toto_win.tkb and the toto_winp.tcl.

 
The interface is trivial. Either you are constructing the "order_list". It is the list that you see in the "order display" (LCD line, or I do not know how to call it). This is easy to do.
 
Go in siglab_ui.tcl, find the place where the array button_calc_command is defined. Look at the abs definition :
 
        set button_calc_command(0,0)  
                 {set event "abs("; set order_list [add_order_list $order_list "abs("];}
        set button_calc_text(0,0) abs(
 
The button_calc_command is the order that will be executed when clicking on the button. Here we see that :
           set event "abs("; # print abs( on the bottom line of the siglab interface
           set order_list [add_order_list $order_list "abs("]; # append to order_list the order abs(.
 
The 0,0 in button_calc_text(0,0) means that this is the button 0 of the panel A. The numbering is :
0 6
1 7
2 8
3 9
4 10
5 11
And panel A is 0, B is 1, C is 2, ...
 
The whole trick is that when you click on OK, the order list is processed, and all Si signals are replaced by loM(i)('matrix')(start_indice:end_indice), and few other things are substituted ( x => * or .*, / => / or ./, ...). After that an ScilabEval "work_out("$changed_list",$work);", create a new signal of type calc. And "la messe est dite" (or that's it in english).
 
OK, but if you do not construct an order_list, it is fairly different. Basically, I would say that "you are on your own", create an order and send it to Scilab with ScilabEval. This is not very difficult, if you are creating a matrix_file signal.
 
Most of the time, you create an interface allowing you to get some parameters (signal index, parameters) in tcl, then create a scilab function that will end with new_signal_from_matrix_file, and then when clicking on ok, you call your function with the parameters you currently have in the interface. You may look at the cspectd user interface (cspect_winp.tcl ?), and to the function in math_tkdf.sci. Fair warning, it is easy to do, I never said it is fast to do. The creation of a graphical interface, even the smallest takes at least 1 to 3 hours. So as usual, start from something existing.
 

5.1. Example

We are going to add a toto function in math_tkdf.sci and the corresponding button in the E panel (file siglab_ui.tcl). toto is a function that adds 1 to all samples of a signal.
 
The text of the toto function is the following :
 
        function [result]=toto(input_matrix);
                debug_function(DEBUG_MATH_TKDF,"[result]=toto(input_matrix);");
                result=input_matrix+1;
                leaving_debug();
 
The debug_function allows while debugging to see in which function the program is entering, and when he is leaving. The leaving_debug at the end, is here to ensure that the indentation is good. If you forget it, the indentation will get mad.
Now let us look at the button definition in siglab_ui.tcl. The button will apear on the panel E :
 
         set button_calc_command(3,4)  
           {set event "toto("; set order_list [add_order_list $order_list "toto("];}
         set button_calc_text(3,4) toto(
 

5.2. Customising the interface

If you have in your home directory the files my_siglab.sci and my_siglab.tcl, there are sourced on startup. It means that you may put the piece of code above that was in math_tkdf.sci in ~/my_siglab.sci, and the piece in siglab_ui.tcl (set button....) in ~/my_siglab.sci. These example files are in directory test.
 
###################### TO BE CONTINUED #########################
 
$Id: programmer.txt,v 1.1.1.1 2002/05/06 13:32:48 arnaud Exp $