Vector & Matrix Maths

[Mike Lobanovsky]

Hi everyone,

OpenGL 3D graphics seems to be a very attractive feature for many potential users of the language. This concerns not only newbies that come and go seeking possible fields of application for their yet undeveloped skills but also some mature individuals like our fellow Fbsl-ers including your humble servant.

Taobert's explorations described HERE have made me think about implementing a compact high-speed math library within the Fbsl source code in order to make some calculations run much faster than they can actually run if performed in a pure Fbsl script.

Computerised 3D virtuality is largely built around fast arithmetical operations with member fields of compound basic entities (user-defined types a.k.a. structures in non-Basic languages) which can be described in Fbsl, for example, as follows:

Code: Select all

Type VECTOR3F ' This is a vector (1-dimensional matrix) of three single-precision numeric values
   x As Single
   y As Single
   z As Single
End Type

Type MATRIX2F ' This is a two-dimensional matrix
   m0 As VECTOR3F
   m1 As VECTOR3F
End Type

Type MATRIX3F ' This is a three-dimensional matrix
   m0 As VECTOR3F
   m1 As VECTOR3F
   m2 As VECTOR3F
End Type ' et cetera, et cetera, et cetera ...

Member fields of these UDTs may all be integers, or singles, or doubles. Of course, each one of them can also be represented as a corresponding one- or multi-dimensional array of integers, singles, or doubles respectively. But in practice, it is much more convenient to store them in memory as user-defined types because UDTs are stored contiguously line-by-line while multi-dimensional arrays follow the so-called "column-major" contiguous storage order whereby vertical columns of identical fields of UDT members are stored rather than horizontal rows (lines) of each member's fields. This implies additional index transformation overhead unnecessary in case of UDTs.

In fact, vectors and up to 4-dimensional matrices are normally used to calculate all the translations, rotations, colors, normals and texture coordinates of an object in its 3D world. Vector and matrix mathematics involves addition to, subtraction from, multiplication by, etc. of all or individual member fields with one another, or with member fields of other matrices, or with user-supplied numeric literals. This functionality is not hardcoded into the OpenGL dll and is the responsibility of end-user's code.

In Fbsl, this implies e.g. the following explicit calculations:

Code: Select all

Function VectorMultiply(vector1, vector2)
   Dim ret As VECTOR3F
   ret.x = vector1.x * vector2.x
   ret.y = vector1.y * vector2.y
   ret.z = vector1.z * vector2.z
   Return ret
End Function

Fbsl does it perfectly but the virtual machine's speed is insufficient to match the potential speed of OpenGL's low-level code. In fact, OpenGL idles while the virtual machine executes its commands to dimension the local variable, read in the parameters, evaluate them for silly errors, compute the offsets, perform basic multiplication proper, and assign the return values. Virtual machines are always slower than pure native CPU opcode generated by true compilers.

A clever way would be to move this function entirely to within the Fbsl source code and expose its name as a new Fbsl keyword (intrinsic function).

A still cleverer way would be to make VECTOR3F, MATRIX2I, MATRIX3D etc. intrinsic Fbsl types and allow the following example expressions:

Code: Select all

Dim result As VECTOR3F, vector1 As VECTOR3F, vector2 As VECTOR3F
' ........ actual vector1 and vector2 value assignments follow here
result = vector1 * vector2 ' That's it!

This will be exactly what can be called Vector and Matrix Maths in bold capitals! And it will run real fast because all this could be optimized C-wise in the Fbsl source code or even inlined there with the crooked AT&T assembler GCC supports... And finally, it will preserve a customary, humanly readable form of presentation.

This functionality can be 'mimic'ed' (don't know how to spell this one correctly to make it sound 'mimikd' rather than 'mimisd' , Yankees please help!... ) in the current Fbsl implementations with overloadable operators in class definitions. But declaring vectors and matrices as classes would make the virtual machine run still slower because classes need additional processing. However, this is true for all languages, interpreted or not.

OpenGL-friendliness is not just another fancy feature. It is a necessity today, an indispensable requirement of a mature language. This is what has made Lua and Python what they currently are.

I'm not polling this question yet. This is a serious challenge and I think we should discuss it far and wide beforehand.

[Codeman]

haha i don't stop kicking gerome's ass to make him add thoses opengl based maths functions as normalise(vec3), matrix operations, vectors operations and co. to speed up the opengl apps, it will also reduce the script size, increase it speed and make our life easier as we can also add (i think) mike opengl class with defines in the embeded xml for adding #USE statement, this way we can concentrate to the real thing : the app !

it will also be a marvelous thing i think

[Codeman]

also some functions like float16to32 and float32to16, my recent ubytetofloat() and floattoubyte() can also be stored as xml functions i think

[Mike Lobanovsky]

Thanks for your response Chris,

I think now Gerome's hind parts will be of a deeper blue 'cuz there's two of us kickin'!

The function way is the easy way, but what do you think of the elegant "result = vector1 * vector2"?

[Mike Lobanovsky]

... A still cleverer way would be to make VECTOR3F, MATRIX2I, MATRIX3D etc. intrinsic Fbsl types ...

And a much more cleverer way would be to make VECTOR and MATRIX only as intrinsic Fbsl types and declare them as e.g.

Code: Select all

Dim my_vector As %VECTOR2, my_matrix As !MATRIX3

or

Code: Select all

Dim my_vector As Integer VECTOR2, my_matrix As Single MATRIX3

while the Fbsl parser itself reads in the trailing numeric character to dimension the vector and matrix accordingly...

Also, to preclude (or include) some stupidities (or freestyle originalities), we can also make MATRIX1 an alias (synonym) to VECTOR...

Another one: I think we can even make it sense automatically that if an arithmetic expression implies the entire content of an array of vectors or matrices then such expression as e.g. a hypothetical

Code: Select all

my_matrix_array|| = my_matrix_array|| * 5.0

will make it move the entire loop to the intrinsics and all the matrix elements of the array of matrices will be multiplied (scaled) by 5.0 very fast within the C or AT&T asm code in order to avoid doing it in an Fbsl loop. I've used "||" here because I yet don't know how to elegantly inform Fbsl that I want this statement to be applied to the entire array contents. "{}" are already used for string{i} access and anonymous {val1, val2,...} arrays, [] is used for declaring dynamic arrays, and <> is used for inequality and #Include files paths if I'm not mistaken.

(BTW Chris, pls note here that assigning anywhere throughout your script e.g. an implicit 2.0 or explicit !!2.0 double -- or an expression or function that returns a double or integer -- to a vector/matrix member field which has been declared "As Single" in its vector/matrix UDT type declaration WILL NOT change it from single to double! It will remain single. You cannot change the type of a UDT member field after it has been declared as a specific data type. This is the only case where Fbsl prohibits any conversion and will cast the incoming value to its proper data type representation. Unless, of course, the type declaration has explicitly declared this member field to be a variant, i.e. no data type specifiers (%,!,!!,$) or "As" declarations have been used. So you may safely get rid of at least 80 per cent of your unnecessary type casts NOW!)

Any more ideas?

[Codeman]

in fact there is a bunch of functions that can be added for vectors and matrix, also for triangle normals calculations etc...

i think you can add the base ones , and i could help testing if needed

[Mike Lobanovsky]

Chris,

Do you imply that you want it all implemented as separate functions?

Do you realize HOW MANY functions do we need to add? Personally, I am not happy with the idea of having another half-hundred function names added to the vocabulary...

If we implement it as genuine arithmetics, we will need to only introduce two special intrinsic (OR EVEN ONE - MATRIX ONLY WITH VECTOR BEING AN ALIAS TO MATRIX1!!!) data types and perhaps a way to indicate that we want the entire array contents to be processed, perhaps some keyword like "DoAll" or special symbol like tilde or some funny brackets (or double brackets [[]]) or what not... And all the paraphernalia of functions proper will be hidden inside Fbsl itself!

Please think twice and please re-read my message above before you take you final decision...

[Codeman]

sure vectex1 * vertex2 could be usefull, i don't know yet what i could use it for lol

[Mike Lobanovsky]

He-he Chris,

This is just an example, a special case of one-dimensional matrix multiplication... Ever heard of glMultMatrix()? It multiplies your matrices with OpenGL's internal projection, modelview, color and texture matrices. Ever cast a glance into my Q3 scripts? You can find it there. You can also find my box-in-frustum culling routines that require 4x4 matrix multiplication. At least one of the scripts includes an equivalent Fbsl prototype implementation commented out that requires around 4K bytes worth of unrolled Fbsl For/Next loops to be able to display anything at a rate of at least one frame per second. Asm does it at 500FPS with culling being done in the renderer callback and not in the data handler callback so no tricks and no fooling around -- "look, no hands...":

Code: Select all

Asm MatrixMul4(%proj, %modl, %clip) ' a,b,c - pointers to matrices
' ======================================================
' Suggested by Charles Pegge at http://www.jose.it-berater.org/smfforum
' ======================================================
#Define   proj   dword ptr[esp + 4]
#Define   modl   dword ptr[esp + 8]
#Define   clip   dword ptr[esp + 12]

.data
@cell   ' row PROJ * column MODL
fld   dword ptr[ecx]
fmul   dword ptr[edx]
fld   dword ptr[ecx + 16]
fmul   dword ptr[edx + 4]
fld   dword ptr[ecx + 32]
fmul   dword ptr[edx + 8]
fld   dword ptr[ecx + 48]
fmul   dword ptr[edx + 12]
faddp   st(1), st
faddp   st(1), st
faddp   st(1), st
fstp   dword ptr[eax]
add   eax, 4
add   ecx, 4
ret

@column
call   cell
call   cell
call   cell
call   cell
add   edx, 16
sub   ecx, 16
ret

.code
mov   ecx, proj
mov   edx, modl
mov   eax, clip

call   column
call   column
call   column
call   column

ret

#Undef   proj
#Undef   modl
#Undef   clip
End Asm

Of course, we will not be able to avoid a couple of functions for which there are no corresponding arithmetic symbols, like e.g. Normalize() or Normal() and DotProduct() or simply Dot() but they are very few indeed as compared to many dozens if we go the other, pure function, way... But we could also introduce some special symbols for them without the need to expand the keyword vocabulary which is rather broad already as it is...