GOODS - Fequently Asked Questions
GOODS core
Using
GOODS embedded server in a
client application
To integrate the GOODS embedded server into your client application
simply add the following lines at the appropriate positions and link
against the server lib, e.g. srvrmfc.dll
#include "C:/Programme/Development/goods/inc/confgrtr.h"
start_goods_server( NameODBCfg );
stop_goods_server();
If you link against static libraries for a Windows32 console
application (client.lib, server.lib) you can use process sockets for
communication between the client and the server thread. To enable using
process sockets simply prepend the server name in the database
configuration file by a '!'
Process sockets could not be used in case of a Win32 MFC application if
you link against shared MFC DLLs. In that case local sockets are being
used. To enable local sockets simply use 'localhost' as server name in
the database configuration file.
Installing
and starting goodsrv.exe as
NT service
Precondition: the supplied user account (prefer the local
administrator) must have the right to start as
a service on the target machine.
To add the right to start as a service do the following:
On german versions of Windows 2000:
Start -> Einstellungen -> Systemsteuerung -> Verwaltung ->
Lokale Sicherheitsrichtlinie -> Lokale Richtlinien -> Zuweisen
von Benutzerrechten
-> Double Click on "Als Dienst anmelden" -> Hinzufügen -> add
user account
See on english versions of Windows 2000:
Start -> Settings -> Control Panel -> Administrative Tools
-> Local Security Policy -> Local Polies -> User Rights
Assignments -> Double click on "Log on as a service" -> add user
account.
1. Copy instsrv.exe and srvany.exe in path of goodsrv.exe,
e.g. C:\BeGaze\ODB
2. Install svrany.exe as service supplying an appropriate name, user
account (prefer local admin, has to be able to start as service! ) and
user PWD
instsrv <service_name> <path>\srvany.exe
-a <domain\account> -p <password>
eg. instsrv GOODSsvr C:\BeGaze\ODB\srvany.exe -a
SMI_GMBH\or -p password (for a domain user account)
eg. instsrv GOODSsvr C:\BeGaze\ODB\srvany.exe -a
.\administrator -p password (for local administrator account; do
not overlook the dot in ".\admini...")
3. Run regedit.exe and browse to the key
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\GOODSsvr or better
(?)
HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\GOODSsvr
4. Create a new key "Parameters" under key as mentioned above
5. Click on the key "Parameters" and create a string value and name it
"Application"
6. Double click on this value and enter the full path to goodsrv.exe
with the full path
to the database config file in quotation marks, e.g.
C:\BeGaze\ODB\goodsrv.exe
"C:\BeGaze\ODB\begazeODB"
7. Finally start the newly created service via the contol applet or
type on command line: net start GOODSsvr
8. You may not move the directory and files mentioned
above to another location after these installation steps!
Dumping objects from a database
- Start GOODS browser application: browser
odb_cfg_file.cfg
- The browser prompt appears.
- Type the storage identifier followed by colon sign, e.g. 0: at
the prompt to dump the unique root object
>>
0:
Object
"GoodsRoot" 0:10000
{_lstExperiments=0:10001,
_lstFiles=0:10002}
- Now you can easily navigate through the tree of objects by typing
the appropriate object ids shown afte the variable name resp. the
storage id:
>>
10001
Object
"Experiments" 0:10001
{B_tree={set_owner={first=0:10003,
last=0:10003, obj=0:0, n_members=1}, root=0:10004, height=1, n=146},
_iterating=0}
Dumping classes from a database
- Start GOODS browser application: browser
odb_cfg_file.cfg
- The browser prompt appears.
- Simply type the class numbers, beginning with 2 for the unique
root object to dump class definitions.
>> 2
class
GoodsRoot {
ref _lstExperiments;
ref _lstFiles;
}
>> 3
class
Experiments {
...
}
Setting the cluster size for fetching objects from server
When an object is requested by the client, the server includes
OR> in his reply all objects which satisfy the three following conditions:
OR> 1. the object is directly accessible from the requested object,
OR> 2. the client doesn't have this object,
OR> 3. the total size of all objects in the reply is limited to some
OR> predefined constant."
C++ Interface
Using GOODS (client and /or server) in a MFC-Application
If you want to use GOODS in an application using MFC as a shared DLL
you have to use "clntmfc.dll" and/or "srvrmfc.dll" as mentioned in
readme.htm, chapter Installation
of GOODS.
Here some additional hints for using GOODS in this configuration:
1.
Since the MFC application framework doesn't like if another files
include windows.h like \goods\inc\stdinc.h does, you have to
change the order of includes:
first include the MS files, e.g. stdafx.h, afx.h and then goods.h.
If it is not possible you could change the file \goods\inc\stdinc.h from
#if defined(_WIN32)
#include <windows.h>
#ifndef __MINGW32__
#pragma warning(disable:4800 4355
4146 4291)
#endif
#endif
to
#if defined(_WIN32)
#ifdef _AFXDLL
#include "afx.h"
#else
#include <windows.h>
#endif
#ifndef __MINGW32__
#pragma warning(disable:4800 4355
4146 4291)
#endif
#endif
But this works only if you use MFC as shared DLL.
If
you use the static library version _AFXDLL will
not be defined.
2.
Disable the MS DEBUG_NEW
In classes generated by the MS application framework you find the
following code to introduce a special new operator for the debug mode:
#ifdef _DEBUG
#define new DEBUG_NEW
This new operator will produce conflicts if you want to create GOODS
persistent classes. So be sure that the line above will be commented
out.
ref< String > as class member
If your class contains an attribute in this form
ref< String > _name;
It is a good idea to check the ref<> to the string before
returning the content, e.g.
return !_name.is_nil() ?
_name->get_text() : "";
How can I modify the size of the client object cache?
The following statement change primary object ache size to 4Mb and
frequently used object cache size to 1Mb.
optimistic_scheme.set_cache_limit(4*1024*1024,
1024*1024);
You can use refernce to any metaobject defined in GOODS, because in
current implementation all metaobjects share the same cache manager.
E.g.
ref<
PersistentClass> pPersObj;
{
w_ref< object > pinned( pPersObj );
pinned.get()->mop->set_cache_limit(
500*1024*1024, 20*1024*1024 );
}
How can I delete a reference to a persistent object to decrement
the usage counter of the object?
Assign NULL to the reference:
ref< Myclass > pMyClass;
pMyClass = NULL;
If pMyClass is the last reference to the object pointed to (no other references to the object exist) the garbage collector will delete the object.
It is save to create persistent members in constructors of
persistent classes?
Example:
PersClass::PersClass(
class_descriptor& desc ) : BaseClass( desc )
{
...
_images =
array_template< ref< Image > >::create();
...
}
This code is ok. You should only avoid to assign references to newly
created class in its constructor to components of some other persistent
objects.
Shall the sequence of members in method describe_components() the
same a in class definition?
No, it is not critical. FIELD macro in any case calculates offset to
the
field.
I registered an instantiation of array_template with REGISTER macro
than with REGISTER_TEMPLATE macro using VC++ 6.0 SP3. Will this
probably cause a side effect?
No, REGISTER_TEMPLATE macro were introduced only because some compiler
doesn't allow to register template class as normal class.
Using the C++ integral type 'long' for member variables of
persistent classes causes compiler errors. Why?
Goods does not support the type long because it is not compatible
across supported platforms. The problem with using long and unsigned
long type is that representation of this type is very machine
dependent (at 32-bit architecture it is 32bit, at 64-bit platform -
64-bit integer). GOODS defined decribe_field functions for all its
scalar types (int1,...). But at 32-bit platform "long" type is not used
at all. And athough representation of int and long is the same at
32-bit platform, compiler still treat them as two distinct types.
and it cause ambiguity problems.
What have I to do to prepare a thread for accessing the database?
a) If the thread shares the same transaction like the main application
there's nothing special to do.
b) If the thread runs in his own isolated transaction you have to call
database::attach() and database::detach().
Here's an sceleton thread function:
void task_proc
store_image_cluster( void* arg )
{
//
Attach current thread to the database and open the client
database db;
db.attach();
if ( db.open( ((const char*)arg) )
) // odb cfg file
name
{
// Get and init root object
ref< GoodsRoot > pGoodsRoot;
db.get_root( pGoodsRoot );
pGoodsRoot->initialize();
// do something in the thread routine
// ...
db.close();
}
db.detach(); // detach our thread from database
}
Is there a way to
flush new and modified objects to disk (to server) to free memory?
No there's no way to flush objects to disk except by commiting the
current transaction.
How do I correctly register a persistent class which is derived
from a template class?
Let's say we have the following class definitions:
template<class
T>
class BTree :
public B_tree
{
public:
BTree( class_descriptor& desc = self_class );
METACLASS_DECLARATIONS(BTree, B_tree);
};
typedef BTree< PersistentClass > BTreeOfPersistentClass;
class UseBTree : public object
{
ref< BTreeOfPersistentClass
> myBTree;
}
...
Correct
registering of myBTree:
REGISTER(myBTree,
B_ree, pessimistic_repeatable_read_scheme);
Not
correct registering of myBTree. Will
cause compiler error because typedef does not introduce a new type but
an alias.
REGISTER(myBTree,
B_tree, pessimistic_repeatable_read_scheme);
How do I correctly define a template class which is derived from a
persistent capable template class?
Base
template class definition:
template<class
T>
class BTree :
public B_tree
{
public:
BTree( class_descriptor& desc = self_class );
METACLASS_DECLARATIONS(BTree, B_tree);
};
a) Either introduce a typedef for the base template class and use in
the declaration of the derived class:
typedef
BTree< Event > BaseCls;
template<class
T>
class
BTreeEvt : public BaseCls
{
public:
BTreeEvt( class_descriptor& desc = self_class );
METACLASS_DECLARATIONS(BTreeEvt, BaseCls);
};
or
b) call the macro for meta class declaration in the derived class
as follows:
template<class T>
class BTreeEvt : public BTree<T>
{
public:
BTreeEvt( class_descriptor& desc = self_class );
METACLASS_DECLARATIONS(BTreeEvt, BTree<T>);
};
Can I use objects created with a C++ application in a Java
application?
Yes, with some restrictions.
For example, Java String field is not the same as C++ char* (in C++
there is special wstring_t type whichcan be mapped to Java String
type). also B-Tree implementation in C++ and Java are completely
different. But it is possible to write classees whjich can be accessed
both from Java and C++.
For further informations please see file readme.htm, hyperlink "GOODS
interface for the Java language" and hyperlink "Compatibility with C++".
How can I pin a persistent object in memory to avoid the client
cache manager to can throw it away?
You have to pin the object in the client cache using the specialized
smart pointer class r_ref<> for read only access and
w_ref<> for write access.
The persistent object gets pinned during the call af the ctor and gets
un-pinned during the call of the dtor.
Example:
ref<
PersistentClass > pPersObject;
...
{
r_ref< object > pinned( pPersObject
); // pin pPersObject in
client cache
if ( dynamic_cast <PersistentClass*> (
pinned.get() ) )
{
// do something with pinned pPersObject
}
} // pPersObject gets
unpinned due to calling of pinned' dtor
How can I savely get RTTI of persistent object?
Simply pin the
object in client cache.
How can I perform an
online database backup?
a) Running Goods-Server as standalone process (goodsrv.exe):
Simply run the following command via the server's command console:
backup <path/file | raw device>
Data can be restored by running the following command
restore <path/file | raw device>
Restore is possible only for not opened database. As far as
goodsrv always implicitly open database at startup, you will have
first to close it using "close" command (and of course to close all clients connected to the database). Then you can execute
"restore" and open database.
b) Running Goods-Server as embedded server in client process (e.g. C++ application):
simply call the function "backup_odb" and provide a file name and wether the function shall wait for completing the backup process (flag notify)
backup_odb( filename, notify );
For restoring the database you have to close the client connection first by calling the method
database:close().
Then you can call the function "restore_odb( filename )" providing the name of the desired backup file.
This function closes the database server implizitly.
After this you can re-open the database client and resume your work.
What's the idea behind the fragmentation of the BLOB into small
blocks?
There are tow main reasons:
1. Avoid reallocation of large objects when data is appended to the
blob.
2. Make it possible to extract BLBO in incremental way (part-by-part).
While one part is handled by application, next can be laoded.
What's about the the block size for BLOBs?
Larger block size leads to better performance but also increase
size of memory lost as a result of fragmentation.
Also fetching large segment can take significant time which cause
application delays.
Regarding chapter optimization of object loading of readme.htm:
"When an object is requested by the client, the server includes in his reply all objects which satisfy the three following conditions:
- The object is directly accessible from the requested object
- the client doesn't have this object,
- the total size of all objects in the reply is limited to some
predefined constant.
What does '1.' exactly mean?
a) The requested object contains a smart pointer to the directly accessible object.
b) the object is embedded by the requested.
What does '3.' exactly mean?
It is "server.cluster_size" parameter which can be set in goodsrv.cfg.
Incresing the size of this parameter will not always increase performance or can event decrease it
(because in this case object which are not needed will be transferred).
If e.g. sizeof( requested object) + sizeof( first referenced object) + embedded object(s) is smaller than the "
server.cluster_size" a potentionaly second referenced object will not be transferred to the client.
This is also true for a referenced B_tree object: as far as the cluster size is not reached the B_tree first pages will be included in the cluster of objects.
When should a optimistic scheme and a pessimistic scheme be used?
a) optimistic scheme:
Optimistic scheme should be used only when it is possible restart transaction. Because with optimistic scheme object access conflict can be detected during transaction commit which cause rollback of current transaction. Application should be ready that transaction can be failed and somehow handle this situation. Look for example at guess application - it uses optimistic locking scheme.
b) pessimistic scheme:
If loosing modification is not acceptable, then you should use one of pessimistic protocol. In this case object will be locked before usage. Pessimistic protocol guarantee that there will be no access conflict during transaction commit. But using pessimistic protocol can cause deadlocks.
If you have hierarchical object structure (all leaves are accessed through the roots), then it is reasonable to use pessimistic protocols for hierarchy roots (to avoid conflicts) and optimistic - for children nodes (to reduce locking overhead).
Are changes of persistent objects visible beyond context of threads?
Yes, they are if the threads share the same transaction. Otherwise modified object will be see by all threads and processes
after transaction commit.
What happens if the multitasking library gets initialized twice
(task::initialize())?
task::initialize
should be called only once in the program. So as far as it is
done in start_goods_server, it should not be done in constructor of
GoodsODBMS. It will cause no problems with Windows multitasking
(because initialize() do nothing here), but can cause problems with
portable and pthread based multitasking.
Where must the file "goodsrv.cfg" resist in case of embedded
server?
In current (working) directory.
Are there differences in performance between running embedded
server or separate server running on the same machine?
There should be no difference. In both cases client and server are connected using sockets. But it is critical whether standard Windows sockets or GOODS local
sockets are used. GOODS local sockets provide about 10 time advantage in performance, especially for small objects.
When does an persistent capabable object become persitent?
GOODS implements persistence on reachabilty approach. If you create an
new object which is derived from other persistent class it is
transient, but "persistent capable". This object automatically
becomes persistent when it is referenced from some other persistent
object.
If there's no reference from other persistent object it will be deleted
by the GC.
When will new objects be moved from the client cache to the server?
When transaction is committed.
Is there a way to flush the cache programmatically?
No, it is not possible to manually control cache.
What is the exact meaning of ODB files (map, idx, odb, log)?
MAP - memory allocation bitmap
IDX - object index. In GOODS object are reference through extra level
of indirection, so OID is used as index in IDX file, each element of
which contains object size, class id and offset in the storage.
ODB - object database
LOG - transaction logfile
Which database files should I save for backup issues?
All files.
What are the limits for the size of a Goods database?
The size of the ODB file is limited by the operating system resp. the
file system.
GOODS also supports work with larger files. Size of database is mostly
limited by size of other files (MAP, IDX, ...) which should fit in main
memory to provide good performance.
When will the garbage collector do it's work?
Garbage collector is periodically started at the server. There are
several criteria of GC initiation which can be controlled through
goodsrv parameters: size of allocated object size last GC, time spent
since last GC, activity of server,...
May I embed an persistent object into another persistent object?
No, an object of persistent class (like B-Tree) can not be embedded in
another
persistent class! It be access
only by reference (or you can derive your
own class from it).
Is there a way to export and import objects from / into a database?
No, currently not because of ambiguousness reasons if only a subset off
all objects are exported.
Does GOODS support virtual base classes for persistent classes?
No, virtual base classes are not supported by GOODS. It is intended
that "object" base class is always in the front of the object.
And it is not true for classes with virtual bases. Instead of
class A :
virtual object {}
class B :
virtual object {}
class C : A,
B {}
You can try the
following:
class Abase {}
class A :
object, Abase {}
class Bbase {}
class B :
object, Bbase {}
class Cbase :
Abase, Bbase {}
class C :
object, Cbase {}
When does the garbage collector delete an object?
An object gets deleted when there are no more references to it in any
client or from any other persistent object in the storage.
It is possible to explicitly begin and end a transaction?
You can use begin_transaction/end_transaction
method of metaobject you are using.
This methods just increment/decrement nested transaction counter.But you can
also make object responsible for editing object also persistent capable. In this case invoking for example
"Edit" methods which will show
document to user and do not return control until user press Save or Cancel button, will be part of one
transaction.
Whats's the advantage of a static create method compared to the
normal constructor of a persistent class?
Create method is used to simplify instantiation of objects with varying
size (in this case size of the object should be passed to operator new
and constructor of object class). So create method just path this
varying size parameter in two targets.
What happens if delete will be called on pointer to transient
object of persistent class?
You should not explicitly delete object!
Should atomic types be casted to GOODS types and vice versa in e.g.
getter/setter methods?
It is not necessary, but desired. At least it prevent you from many
compiler warnings.
How can I access the operator [] if I have a ref<> to an
array_template?
ref< array_template<
PersistentClass > > myArrayTemplate;
a) ( *( ( myArrayTemplate.operator->()
).operator->() ) )[i];
b) myArrayTemplate->getat( i );
Is there a way to
realize the dereference * operator for ref<> ?
No, it is safe to access persistent object only when it is pinned by
smart pointer. And dereference operator will have to return normal
reference and so object will be not protected.
Would it be a problem if I declare a member variable as enum?
It is up to compiler how to represent enums. It may use signed or
unsigned, short, integer or byte type. From this point of view, it is
safer to explicitly specify sizeof of
the field. Otherwise it can happen that after adding few new items to
the enum, compiler will change its representation and all object
instances of this class has to be reformatted. Also it may cause
overloading problem (there will be no strict match).
When shall the overloaded new operator new (self_class,
strlen(name)+1) be used?
You need to use this new operator only for classes with varying length,
i.e.:
class
Foo {
int x;
int y;
char
moreData[1];
};
In this class size of moreData array is determined at object creation
time. In this case you will have to use operator new with extra
parameter specifying size of varying part. If you allocate more bytes
then sizeof(Foo), you can write in the last field specified number of
bytes.
In which cases should a variable with varying length be preferred
rather than a ref<String>?
Varying strings are better choice because them are more flexible (class
can have only one varying component but multiple string components).
But originally string are not supported in GOODS. Also varying
components are used to represent arrays. Comparing with
reference, varying components are more efficient because we have one
object instead of two. So fetching of such object is almost two times
faster. Also it keeps less space on the disk.
Varying components were mostly introduced for storing arrays in GOODS
and should not widely be used in application. Instead of it you should
use such classes as ArrayOfXXX, String, ...
May transient classes embedded in persistent classes?
Yes. See explanation in Goods_Quickstart.html
Is it save to access member variables of a persistent class through
getter method?
Let's say we have:
char const*
get_name() { return _name; }
It is not save. And doesn't matter whether _name is varying field,
array with fixed length or string. The problem is that GOODS object
cache manager can replace persistent object at any moment of time when
it is not accessed. And object considered as not accessed when its
access counter is 0. Access counter is incremented by metaobject before
access to the object and decremented after end of access. Now consider
that you have reference to some persistent object and invoke get_name
method:
ref<Foo>
fref;
fref->get_name();
Before invocation of get_name method, access counter is incremented,
and after return from this method, access counter is decremented and
becomes 0. Now object cache LRU replacement algorithm can replace this
object. In this case pointer returned by get_name function will not be
valid any more.
It means that all access to the persistent object fields should be
within instance method of this object and if you want to return some
non-scalar value of object field, you should create copy of this value:
string
get_name() { return string(_name); }
What's the advantages / disadvantages of B_tree, H_tree,
hash_table, R_tree?
B-Tree is more classical approach which is used in most DBMSes.
R-Tree is used for searching spatial data (rectangles, points, ...).
H-Tree is hash table with hierarchical structure. It doesn't
preserve elements order.
So B-Tree is more universal because it allows range queries.
H_Tree also use internal object to represent collision chains. The only
difference
with B-Tree is that them are not visible to the user. But there is
still extra level of indirection and so H-Tree has no advantage in
performance in this aspect. B-Tree provides better flexibility: range
queries, sorting, and ability to include object in more than one B-Tree.
Hashtable is more preferable if you use only search with strict
comparision and do not use range queries or sorting. Also it is more
efficient if you can estimate number of objects which will be placed in
container.
What's about the size of the hash table?
Size of hash table is just maximal number of entries containing
collision chains. You can certainly place in H-Tree significantly more
elements than hash size. But in this case collision chain in average
will contain more than one element and so search will be slower.