GOODS  -  Fequently Asked Questions


GOODS core

Using GOODS embedded server in a client application

To integrate the GOODS embedded server into your client application simply add the following lines at the appropriate positions and link against the server lib, e.g. srvrmfc.dll
#include "C:/Programme/Development/goods/inc/confgrtr.h"
start_goods_server( NameODBCfg );
stop_goods_server();

If you link against static libraries for a Windows32 console application (client.lib, server.lib) you can use process sockets for communication between the client and the server thread. To enable using process sockets simply prepend the server name in the database configuration file by a '!'

Process sockets could not be used in case of a Win32 MFC application if you link against shared MFC DLLs. In that case local sockets are being used. To enable local sockets simply use 'localhost' as server name in the database configuration file.


Installing and starting goodsrv.exe as NT service

Precondition: the supplied user account (prefer the local administrator) must have the right to start as a service on the target machine.

To add the right to start as a service do the following:

On german versions of  Windows 2000:
Start -> Einstellungen -> Systemsteuerung -> Verwaltung -> Lokale Sicherheitsrichtlinie -> Lokale Richtlinien -> Zuweisen von Benutzerrechten
-> Double Click on "Als Dienst anmelden" -> Hinzufügen -> add user account

See on english versions of Windows 2000:
Start -> Settings -> Control Panel -> Administrative Tools -> Local Security Policy -> Local Polies -> User Rights Assignments -> Double click on "Log on as a service" -> add user account.

1. Copy instsrv.exe and srvany.exe in path of goodsrv.exe,
   e.g. C:\BeGaze\ODB

2. Install svrany.exe as service supplying an appropriate name, user account (prefer local admin, has to be able to start as service! ) and user PWD
   instsrv <service_name>  <path>\srvany.exe -a <domain\account> -p <password>
   eg. instsrv GOODSsvr C:\BeGaze\ODB\srvany.exe -a SMI_GMBH\or -p password  (for a domain user account)
   eg. instsrv GOODSsvr C:\BeGaze\ODB\srvany.exe -a .\administrator -p password  (for local administrator account; do not overlook the dot in ".\admini...")
 

3. Run regedit.exe and browse to the key
   HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\GOODSsvr or better (?)
   HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\GOODSsvr

4. Create a new key "Parameters" under key as mentioned above
 
5. Click on the key "Parameters" and create a string value and name it "Application"

6. Double click on this value and enter the full path to goodsrv.exe with the full path
   to the database config file in quotation marks, e.g.
   C:\BeGaze\ODB\goodsrv.exe "C:\BeGaze\ODB\begazeODB"
 
7. Finally start the newly created service via the contol applet or
     type on command line: net start GOODSsvr

8. You may not move the directory and files mentioned above to another location after these installation steps!



Dumping objects from a database

  1. Start GOODS browser application: browser odb_cfg_file.cfg
  2. The browser prompt  appears.
  3. Type the storage identifier followed by colon sign, e.g. 0: at the prompt to dump the unique root object
    >> 0:
    Object "GoodsRoot" 0:10000
    {_lstExperiments=0:10001, _lstFiles=0:10002}
  4. Now you can easily navigate through the tree of objects by typing the appropriate object ids shown afte the variable name resp. the storage id:
    >> 10001
    Object "Experiments" 0:10001
    {B_tree={set_owner={first=0:10003, last=0:10003, obj=0:0, n_members=1}, root=0:10004, height=1, n=146}, _iterating=0}

Dumping classes from a database

  1. Start GOODS browser application: browser odb_cfg_file.cfg
  2. The browser prompt  appears.
  3. Simply type the class numbers, beginning with 2 for the unique root object to dump class definitions.
    >> 2
    class GoodsRoot {
            ref _lstExperiments;
            ref _lstFiles;
    }
    >> 3
    class Experiments {
            ...
    }

Setting the cluster size for fetching objects from server


When an object is requested by the client, the server includes
OR> in his reply all objects which satisfy the three following conditions:

OR> 1. the object is directly accessible from the requested object,
OR> 2. the client doesn't have this object,
OR> 3. the total size of all objects in the reply is limited to some
OR> predefined constant."



C++ Interface

Using GOODS (client and /or server) in a MFC-Application

If you want to use GOODS in an application using MFC as a shared DLL you have to use "clntmfc.dll" and/or "srvrmfc.dll" as mentioned in readme.htm, chapter Installation of GOODS.
 

Here some additional hints for using GOODS in this configuration:

1.
Since the MFC application framework doesn't like if another files include windows.h like  \goods\inc\stdinc.h does, you have to change the order of includes:
first include the MS files, e.g. stdafx.h, afx.h and then goods.h.
If it is not possible you could change the file \goods\inc\stdinc.h from

#if defined(_WIN32)
#include <windows.h>
#ifndef __MINGW32__
#pragma warning(disable:4800 4355 4146 4291)
#endif
#endif
to
#if defined(_WIN32)
#ifdef _AFXDLL
#include "afx.h"
#else
#include <windows.h>
#endif
#ifndef __MINGW32__
#pragma warning(disable:4800 4355 4146 4291)
#endif
#endif

But this works only  if  you use MFC as shared DLL. If you use the static library version _AFXDLL will not be defined.


2.
Disable the MS DEBUG_NEW
In classes generated by the MS application framework you find the following code to introduce a special new operator for the debug mode:
#ifdef _DEBUG
#define new DEBUG_NEW

This new operator will produce conflicts if you want to create GOODS persistent classes. So be sure that the line above will be commented out.



ref< String > as class member

If your class contains an attribute in this form
ref< String > _name;
It is a good idea to check the ref<> to the string before returning the content, e.g.

return !_name.is_nil() ? _name->get_text() : "";


How can I modify the size of the client object cache?

The following statement change primary object ache size to 4Mb and frequently used object cache size to 1Mb.

optimistic_scheme.set_cache_limit(4*1024*1024, 1024*1024);

You can use refernce to any metaobject defined in GOODS, because in
current implementation all metaobjects share the same cache manager.
E.g.
ref< PersistentClass> pPersObj;
{
    w_ref< object > pinned( pPersObj );
    pinned.get()->mop->set_cache_limit( 500*1024*1024, 20*1024*1024 );
}


How can I delete a reference to a persistent object to decrement the usage counter of the object?

Assign NULL to the reference:
ref< Myclass > pMyClass;
pMyClass = NULL;

If pMyClass is the last reference to the object pointed to (no other references to the object exist) the garbage collector will delete the object.


It is save to create persistent members in constructors of persistent classes?

Example:
PersClass::PersClass( class_descriptor& desc ) : BaseClass( desc )
{
    ...  
    _images = array_template< ref< Image > >::create();

    ...
}

This code is ok. You should only avoid to assign references to newly created class in its constructor to components of some other persistent objects.


Shall the sequence of members in method describe_components() the same a in class definition?

No, it is not critical. FIELD macro in any case calculates offset to the field.


I registered an instantiation of array_template with REGISTER macro than with REGISTER_TEMPLATE macro using VC++ 6.0 SP3. Will this probably cause a side effect?

No, REGISTER_TEMPLATE macro were introduced only because some compiler doesn't allow to register template class as normal class.


Using the C++ integral type 'long' for member variables of persistent classes causes compiler errors. Why?

Goods does not support the type long because it is not compatible across supported platforms. The problem with using long and unsigned long type is that  representation of this type is very machine dependent (at 32-bit architecture it is 32bit, at 64-bit platform - 64-bit integer). GOODS defined decribe_field functions for all its scalar types (int1,...). But at 32-bit platform "long" type is not used at all. And athough representation of int and long is the same at 32-bit  platform, compiler still treat them as two distinct types. and it cause ambiguity problems.


What have I to do to prepare a thread for accessing the database?

a) If the thread shares the same transaction like the main application there's nothing special to do.

b) If the thread runs in his own isolated transaction you have to call database::attach() and database::detach().
Here's an sceleton thread function:
void task_proc store_image_cluster( void* arg )
{
    // Attach current thread to the database and open the client
    database db;
    db.attach();
   
    if ( db.open(
((const char*)arg) ) ) // odb cfg file name
    {
        // Get and init root object
        ref< GoodsRoot > pGoodsRoot;
        db.get_root( pGoodsRoot );
        pGoodsRoot->initialize();
       
        // do something in the thread routine
        // ...

        db.close();
    }
    db.detach(); // detach our thread from database
}

Is there a way to flush new and modified objects to disk (to server) to free memory?

No there's no way to flush objects to disk except by commiting the current transaction.


How do I correctly register a persistent class which is derived from a template class?

Let's say we have the following class definitions:

template<class T>

class BTree : public B_tree
{
  public:
     BTree( class_descriptor& desc = self_class );

     METACLASS_DECLARATIONS(BTree, B_tree);
};

typedef BTree< PersistentClass > BTreeOfPersistentClass;

class UseBTree : public object
{
    ref<
BTreeOfPersistentClass > myBTree;
}

...

Correct registering of myBTree:
REGISTER(myBTree, B_ree, pessimistic_repeatable_read_scheme);

Not correct registering of myBTree. Will cause compiler error because typedef does not introduce a new type but an alias.
REGISTER(myBTree, B_tree, pessimistic_repeatable_read_scheme);

How do I correctly define a template class which is derived from a persistent capable template class?

Base template class definition:

template<class T>
class BTree : public B_tree
{
  public:
     BTree( class_descriptor& desc = self_class );

     METACLASS_DECLARATIONS(BTree, B_tree);
};


a) Either introduce a typedef for the base template class and use in the declaration of the derived class:


typedef BTree< Event > BaseCls;

template<class T>
class BTreeEvt : public BaseCls
{
  public:
    BTreeEvt( class_descriptor& desc = self_class );

    METACLASS_DECLARATIONS(BTreeEvt, BaseCls);
};

or
b)  call the macro for meta class declaration in the derived class as follows:
template<class T>
class BTreeEvt : public BTree<T>
{
public:
BTreeEvt( class_descriptor& desc = self_class );

METACLASS_DECLARATIONS(BTreeEvt, BTree<T>);
};



Can I use objects created with a C++ application in a Java application?

Yes, with some restrictions.
For example, Java String field is not the same as C++ char* (in C++
there is special wstring_t type whichcan be mapped to Java String
type). also B-Tree implementation in C++ and Java are completely
different. But it is possible to write classees whjich can be accessed
both from Java and C++.
For further informations please see file readme.htm, hyperlink "GOODS interface for the Java language" and hyperlink "Compatibility with C++".

How can I pin a persistent object in memory to avoid the client cache manager to can throw it away?

You have to pin the object in the client cache using the specialized smart pointer class r_ref<> for read only access and w_ref<> for write access.
The persistent object gets pinned during the call af the ctor and gets un-pinned during the call of the dtor.
Example:
ref< PersistentClass > pPersObject;
...
{  
    r_ref< object > pinned(
pPersObject );  // pin pPersObject in client cache
    if ( dynamic_cast <PersistentClass*> ( pinned.get() ) )
    {
        // do something with pinned
pPersObject
    }
} //
pPersObject gets unpinned due to calling of pinned' dtor

How can I savely get RTTI of persistent object?

Simply pin the object in client cache.

How can I perform an online database backup?

a) Running Goods-Server as standalone process (goodsrv.exe):
Simply run the following command via the server's command console:
backup <path/file | raw device>

Data can be restored by running the following command
restore <path/file | raw device>
Restore is possible only for not opened database. As far as
goodsrv always implicitly open database at startup, you will have
first to close it using "close" command (and of course to close all clients connected to the database). Then you can execute
"restore" and open database.

b) Running Goods-Server as embedded server in client process (e.g. C++ application):
simply call the function "backup_odb" and provide a file name and wether the function shall wait for completing the backup process (flag notify)
backup_odb( filename, notify );

For restoring the database you have to close the client connection first by calling the method
database:close().
Then you can call the function "restore_odb( filename )" providing the name of the desired backup file.
This function closes the database server implizitly.
After this you can re-open the database client and resume your work.

What's the idea behind the fragmentation of the BLOB into small blocks?

There are tow main reasons:
1. Avoid reallocation of large objects when data is appended to the
blob.
2. Make it possible to extract BLBO in incremental way (part-by-part).
While one part is handled by application, next can be laoded.

What's about the the block size for BLOBs?

Larger block size leads to better performance but also increase
size of memory lost as a result of fragmentation.
Also fetching large segment can take significant time which cause
application delays.

Regarding chapter optimization of object loading of readme.htm:

"When an object is requested by the client, the server includes in his reply all objects which satisfy the three following conditions:
  1. The object is directly accessible from the requested object
  2. the client doesn't have this object,
  3. the total size of all objects in the reply is limited to some predefined constant.

What does '1.' exactly mean?

a) The requested object contains a smart pointer to the directly accessible object.
b) the object is embedded by the requested.

What does '3.' exactly mean?

It is "server.cluster_size" parameter which can be set in goodsrv.cfg.
Incresing the size of this parameter will not always increase performance or can event decrease it
(because in this case object which are not needed will be transferred).

If e.g. sizeof( requested object) + sizeof( first referenced object) + embedded object(s) is smaller than the "
server.cluster_size" a potentionaly second referenced object will not be transferred to the client.

This is also true for a referenced B_tree object: as far as the cluster size is not reached the B_tree first pages will be included in the cluster of objects.

When should a optimistic scheme and a pessimistic scheme be used?

a) optimistic scheme:
Optimistic scheme should be used only when it is possible restart transaction. Because with optimistic scheme object access conflict can be detected during transaction commit which cause rollback of current transaction. Application should be ready that transaction can be failed and somehow handle this situation. Look for example at guess application - it uses optimistic locking scheme.

b) pessimistic scheme:
If loosing modification is not acceptable, then you should use one of pessimistic protocol. In this case object will be locked before usage. Pessimistic protocol guarantee that there will be no access conflict during transaction commit. But using pessimistic protocol can cause deadlocks.

If you have hierarchical object structure (all leaves are accessed through the roots), then it is reasonable to use pessimistic protocols for hierarchy roots (to avoid conflicts) and optimistic - for children nodes (to reduce locking overhead).

Are changes of persistent objects visible beyond context of threads?

Yes, they are if the threads share the same transaction. Otherwise modified object will be see by all threads and processes
after transaction commit.

What happens if the multitasking library gets initialized twice (task::initialize())?

task::initialize should be called only once in the program. So as far as it is done in start_goods_server, it should not be done in constructor of GoodsODBMS. It will cause no problems with Windows multitasking (because initialize() do nothing here), but can cause problems with portable and pthread based multitasking.

Where must the file "goodsrv.cfg" resist in case of embedded server?

In current (working) directory.

Are there differences in performance between running embedded server or separate server running on the same machine?

There should be no difference. In both cases client and server are connected using sockets. But it is critical whether standard Windows sockets or GOODS local
sockets are used. GOODS local sockets provide about 10 time advantage in performance, especially for small objects.

When does an persistent capabable object become persitent?

GOODS implements persistence on reachabilty approach. If you create an new object which is derived from other persistent class it is transient, but "persistent  capable". This object automatically becomes persistent when it is referenced from some other persistent object.
If there's no reference from other persistent object it will be deleted by the GC.

When will new objects be moved from the client cache to the server?

When transaction is committed.

Is there a way to flush the cache programmatically?

No, it is not possible to manually control cache.

What is the exact meaning of ODB files (map, idx, odb, log)?

MAP - memory allocation bitmap
IDX - object index. In GOODS object are reference through extra level of indirection, so OID is used as index in IDX file, each element of which contains object  size, class id and offset in the storage.
ODB - object database
LOG - transaction logfile

Which database files should I save for backup issues?

All files.

What are the limits for the size of a Goods database?

The size of the ODB file is limited by the operating system resp. the file system.
GOODS also supports work with larger files. Size of database is mostly limited by size of other files (MAP, IDX, ...) which should fit in main memory to provide good performance.

When will the garbage collector do it's work?

Garbage collector is periodically started at the server. There are several criteria of GC initiation which can be controlled through goodsrv parameters: size of allocated object size last GC, time spent since last GC, activity of server,...


May I embed an persistent object into another persistent object?

No, an object of persistent class (like B-Tree) can not be embedded in another
persistent class! It be access only by reference (or you can derive your own class from it).

Is there a way to export and import objects from / into a database?

No, currently not because of ambiguousness reasons if only a subset off all objects are exported.

Does GOODS support virtual base classes for persistent classes?

No, virtual base classes are not supported by GOODS. It is intended that "object" base class is always in the front of the object.
And it is not true for classes with virtual bases. Instead of

class A : virtual object {}

class B : virtual object {}

class C : A, B {}


You can try the following:


class Abase {}

class A : object, Abase {}

class Bbase {}

class B : object, Bbase {}

class Cbase : Abase, Bbase {}

class C : object, Cbase {}

When does the garbage collector delete an object?

An object gets deleted when there are no more references to it in any
client or from any other persistent object in the storage.


It is possible to explicitly begin and end a transaction?

You can use begin_transaction/end_transaction method of metaobject you are using. This methods just increment/decrement nested transaction counter.But you can also make object responsible for editing object also persistent capable. In this case invoking for example "Edit" methods which will show document to user and do not return control until user press Save or Cancel button, will be part of one transaction.


Whats's the advantage of a static create method compared to the normal constructor of a persistent class?

Create method is used to simplify instantiation of objects with varying size (in this case size of the object should be passed to operator new and constructor of object class). So create method just path this varying size parameter in two targets.

What happens if delete will be called on pointer to transient object of  persistent class?

You should not explicitly delete object!

Should atomic types be casted to GOODS types and vice versa in e.g. getter/setter methods?

It is not necessary, but desired. At least it prevent you from many compiler warnings.


How can I access the operator [] if I have a ref<> to an array_template?

ref< array_template< PersistentClass > > myArrayTemplate;

a) ( *( ( myArrayTemplate.operator->() ).operator->() ) )[i];
b) myArrayTemplate->getat( i );

Is there a way to realize the dereference * operator for ref<> ?

No, it is safe to access persistent object only when it is pinned by smart pointer. And dereference operator will have to return normal reference and so object will be not protected.

Would it be a problem if I declare a member variable as enum?

It is up to compiler how to represent enums. It may use signed or unsigned, short, integer or byte type. From this point of view, it is safer to explicitly specify sizeof of
the field. Otherwise it can happen that after adding few new items to the enum, compiler will change its representation and all object instances of this class has to be reformatted. Also it may cause overloading problem (there will be no strict match).

When shall the overloaded new operator new (self_class, strlen(name)+1) be used?

You need to use this new operator only for classes with varying length, i.e.:

class Foo {
int x;
int y;
char moreData[1];
};


In this class size of moreData array is determined at object creation time. In this case you will have to use operator new with extra parameter specifying size of varying part. If you allocate more bytes then sizeof(Foo), you can write in the last field specified number of bytes.

In which cases should a variable with varying length be preferred rather than a ref<String>?

Varying strings are better choice because them are more flexible (class can have only one varying component but multiple string components). But originally string are not supported in GOODS. Also varying components are used to represent arrays.  Comparing with reference, varying components are more efficient because we have one object instead of two. So fetching of such object is almost two times faster. Also it keeps less space on the disk.
Varying components were mostly introduced for storing arrays in GOODS and should not widely be used in application. Instead of it you should use such classes as ArrayOfXXX, String, ...


May transient classes embedded in persistent classes?

Yes. See explanation in Goods_Quickstart.html

Is it save to access member variables of a persistent class through getter method?

Let's say we have:
char const* get_name() { return _name; }

It is not save. And doesn't matter whether _name is varying field, array with fixed length or string. The problem is that GOODS object cache manager can replace persistent object at any moment of time when it is not accessed. And object considered as not accessed when its access counter is 0. Access counter is incremented by metaobject before access to the object and decremented after end of access. Now consider that you have reference to some persistent object and invoke get_name method:

ref<Foo> fref;
fref->get_name();

Before invocation of get_name method, access counter is incremented, and after return from this method, access counter is decremented and becomes 0. Now object cache LRU replacement algorithm can replace this object. In this case pointer returned by get_name function will not be valid any more.

It means that all access to the persistent object fields should be within instance method of this object and if you want to return some non-scalar value of object field, you should create copy of this value:

string get_name() { return string(_name); }

What's the advantages / disadvantages of B_tree, H_tree, hash_table, R_tree?

B-Tree is more classical approach which is used in most DBMSes.
R-Tree is used for searching spatial data (rectangles, points, ...).
H-Tree is hash table with hierarchical structure. It doesn't
preserve elements order.
So B-Tree is more universal because it allows range queries.
H_Tree also use internal object to represent collision chains. The only difference
with B-Tree is that them are not visible to the user. But there is still extra level of indirection and so H-Tree has no advantage in performance in this aspect. B-Tree provides better flexibility: range queries, sorting, and ability to include object in more than one B-Tree.
Hashtable is more preferable if you use only search with strict comparision and do not use range queries or sorting. Also it is more efficient if you can estimate number of objects which will be placed in container.


What's about the size of the hash table?

Size of hash table is just maximal number of entries containing collision chains. You can certainly place in H-Tree significantly more elements than hash size. But in this case collision chain in average will contain more than one element and so search will be slower.