DBreeze Documentation 032

DBreezeDatabaseDocumentation.
(DBreezeversion01.074.20160329doc.version01.031.20160329)
Professional,opensource,NoSql
(embeddedKey/Valuestorage),
transactional,ACIDcompliant,
multithreadeddatabasemanagement
systemfor.NET3.5>/XamarinMONO/
.NETCore.
WritteninC#.
Copyright2012dbreeze.tiesky.com
AlexeySolovyov
<
hhblaze@gmail.com
>
IvarsSudmalis
<
zikills@gmail.com
>
It'safreesoftwareforthose,whothinksthatitshouldbefree.
Please,notifyusaboutoursoftwareusage,sowecanevaluateandvisualizeitsefficiency.
Documentevolution.
Thisdocumentevolvesdownside.Allnewfeatures,ifyouhavereadthebasedocument
before,willbereflectedunderneath.Newevolutionalwaysstartsfromamarkinformat[year
monthday][20120521]foreasysearch.
Evolutionhistory[yyyyMMdd]
20160329DBreeze.DataStructures.DataAsTreeanotherwaytorepresentstoreddata.
20160320Quickstartguide.Customersandorders.
20160304ExampleofDBreezeinitializationforUniversalWindowsPlatform
20140603Storingbyte[]serializedobjects(Protobuf.NET).
20130812InsertkeyoverloadforMasterandNestedtable,lettingnottooverwritekeyifit
alreadyexists.
SpeedingupselectoperationsandtraversalswithValuesLazyLoadingIsOn.
20130811RemoveKey/Valueandgetdeletedvalueandnotificationifvalueexistedinone
round.
20130613Fulllockingoftablesinsideoftransaction.
20130610Restoringtablefromtheothertable.
20130529Speedingupbatchmodifications(updates,randominserts)with
Technical_SetTable_OverwriteIsNotAllowedinstruction.
20121111Alternativetablesstoragelocations.
20121101Addednewiteratorsfortransactionmasterandnestedtables
SelectForwardStartsWithClosestToPrefixandSelectBackwardStartsWithClosestToPrefix.
20121023DBreezelikeinmemorydatabase.Outoftheboxbulkinsertspeedincrease.
20121016SecondaryIndexes.Goingdeeper.Part2.
20121015SecondaryIndexes.Goingdeeper.
20121012Behaviouroftheiteratorswiththemodificationinstructionsinside.
20120922!!!!Important,attachnewDBreezeandrecompileyourproject,ifyouhaveerrors
concerningDateTimeconversionfunctionsreadthedocuarticle.
Storingvirtualcolumnsinthevalue,nullabledatatypesandnullabletextof
fixedlength.
20120905Supportofincrementalbackup.
20120628RowhaspropertyLinkToValue
20120601Storinginsideofarowacolumnofdynamicdatalength.InsertDataBlock
HashFunctionsofcommonusage.Fastaccesstolongstringsandbytearrays.
(
updated20121012
)
.
20120529Nestedtablesmemorymanagement.NestedTableClose(),controllingmemory
consumption.
SecondaryIndex.Directkeyselect.
20120526InsertDictionary/SelectDictionaryInsertHashSet/SelectHashSetcontinuation.
20120525Row.GetTable().
InsertDictionary/SelectDictionaryInsertHashSet/SelectHashSet
20120521FractalTablesstructuredescriptionandusagetechniques.
20120509Basictechniquesdescription
[20120509]
Gettingstarted.
DBreeze.dllcontainsfullymanagedcodewithoutreferencestootherlibraries.CurrentDLL
sizeisaround274KB.Startusingitbyaddingitsreferencetoyourproject.Dontforget
DBreeze.XMLfromReleasefoldertogetVSIntelliSensehelp.
DBreezeisadiskbaseddatabasesystem,thoughitalsocanworklikeinmemorystorage.
DbreezedoesnthavevirtualfilesystemunderneathandresidesallworkingfilesinyourOS
filesystem,thatswhyyoumustinstantiateitsenginebysupplyingafoldernamewhereall
fileswillbelocated.
MainDBreeze
namespace
is
DBreeze
.
usingDBreeze
DBreezeEngineengine=null
if(engine==null)
engine=newDBreezeEngine(@"D:\temp\DBR1")
Its
important
inthe
Dispose
functionofyourapplicationorDLLtocallDBreezeengine
Dispose,tohavegracefulapplicationtermination.
if(engine!=null)
engine.Dispose()
Afteryouhaveinstantiatedtheenginetwooptionswillbeavailableforyou,eithertowork
withthedatabaseschemeortoworkwiththetransactions.
Scheme.
Youdontneedtocreatetablesviascheme,itsneededtomakemanipulationswithalready
existingobjects.
Deletingtable:
engine.Scheme.DeleteTable(stringuserTableName)
Gettingspecifictablesnames:
engine.Scheme.GetUserTableNamesStartingWith(stringmask)
Renamingtable:
engine.Scheme.RenameTable(stringoldTableName,stringnewTableName)
Checkingiftableexists:
engine.Scheme.IfUserTableExists(stringtableName)
Gettingphysicalpathtothefileholdingthetable:
engine.Scheme.GetTablePathFromTableName(stringtableName)
Latermorefunctionswillbeaddedthereandtheirdescriptionhere.
Transactions
InDBreezealloperationswiththedata,whichresidesinsideofthetables,mustoccurinside
ofthetransaction.
Weopentransactionlikethis:
using(vartran=engine.GetTransaction())
{
Pleasenote
,thatits
important
todisposetransactionafterallnecessaryoperationsare
done(usingstatementmakesitautomatically).
Pleasenote
,thatonetransactioncanberunonlyin
one
.NETmanagedthreadandcannot
bedelegatedtootherthreads.
Pleasenote
,thatnestedtransactionsare
not
allowed(parenttransactionwillbeterminated)
Duringintransactionaloperationsdifferentthingscanhappenthatswhywe
highly
recommend
tousetrycatchblocktogetherwiththetransactionand
logexceptions
forthe
futureanalysis.
try
{
{
}
}
catch(Exceptionex)
{
Console.WriteLine(ex.ToString())
}
Tabledatatypes
EverytableinDBreezeisakey/valuestorage.Onthelowlevel,keysandvaluesrepresent
arraysofbytesbyte[].
Onthetoplevelyoucanchooseyourowndatatype,fromallowedlist,tobestoredasakey
orvalue.
TherearesomenotstandarddatatypesinDBreeze,addedforusability,theyareaccessible
insideofDBreeze.DataTypesnamespace.
usingDBreeze.DataTypes
Tabledatatypes.Keydatatypes
Keys
cannot
contain
NULLABLE
datatypes.
Note,thatkeyinthetableisalwaysunique.
Hereisalistofavailabledatatypesforthekey:
Keysdatatypes
byte[]
int
uint
long
ulong
short
ushort
byte
sbyte
DateTime
double
float
decimal
string
thisonewillbeconvertedintobyte[]usingUTF8encoding
DbUTF8
DbAscii
thisonewillbeconvertedintobyte[]usingAsciiencoding
DbUnicode
thisonewillbeconvertedintobyte[]usingUnicodeencoding
char
Valuedatatypes
byte[]
int
int?
uint
uint?
long
long?
ulong
ulong?
short
short?
ushort
ushort?
byte
byte?
sbyte
sbyte?
DateTime
DateTime?
double
double?
float
float?
decimal
decimal?
string
DbUTF8
DbAscii
thisonewillbeconvertedintobyte[]usingAsciiencoding
DbUnicode
thisonewillbeconvertedintobyte[]usingUnicodeencoding
bool
bool?
char
char?
Andsomemoreexoticdatatypeslike:
DbXML<T>
DbMJSON<T>
DbCustomSerializer<T>
theyareusedforstoringobjectsinsideofthevalue,wewilltalkaboutthemlater.
Tableoperations.Insertingdata
Alloperationswiththedata,exceptoperationswhichcanbedoneviascheme,mustbedone
insideofthetransactionscope.Bypressingtran.intellisensewillgiveyoualistofall
possibleoperations.Westartfrominsertingdataintothetable.
publicvoidExample_InsertingData()
{
tran.Insert<int,int>("t1",1,1)
tran.Commit()
}
}
Inthisexamplewehaveinserteddataintothetablewiththenamet1.
Table
willbe
createdautomatically
,ifitdoesntexist.
Keytypeforourtableisint1,valuetypeoftableisalsoint(also1).
AfteroneorseriesofmodificationsinsideofthetransactionwemusteitherCommitthemor
Rollbackthem.
Note,
Rollback
functionwill
automaticallyrun
inthe
transactionDispose
function,soall
notcommittedmodificationsofthedatabaseinsideoftransactionwillbeautomatically
rolledback.
Youcanbesurethatthismodificationwillnotbeappliedtothetable,butneverthelessempty
tablewillbecreated,ifitdoesntexistbefore.
{
//NOCOMMIT
}
Wedontstoreinthetabledatatypes,whichyouassumemustbethere,tableholdsonly
bytearraysofkeysandvaluesandonlyontheupperlevelacquiredbyte[]willbeconverted
intokeysorvaluesoftheappropriatedatatypesfromgenericconstructions.
Youcanmodifymorethenonetableinsideofthetransaction.
tran.Insert<uint,string>("t2",1,hello)
tran.Commit()
//or
//tran.Rollback()
tran.Insert<uint,string>("t2",2,world)
tran.Commit()
CommitsandRollbacks
UsedCommitorRollbackwillbeappliedtoallmodificationsinsideofthetransaction.If
somethinghappensduringCommitalldatawillbeautomaticallyrolledbackforall
modifications.

TheonlyacceptablereasonforRollbackfailcanbethedamageofthephysicalstorage,and
exceptionsintherollbackprocedurewillbringdatabasetothenotoperablestate.
DBreezedatabase,afteritsstart,checkstransactionsjournalandrestorestablesintotheir
previousstate,sothereshouldbenoproblemswiththepowerlossoranyotheraccidental
softwareterminationinanyprocessexecutionpoint.
DBreezedatabaseisfully
ACID
compliant.
Commit
operationisalwaysvery
fast
andtakesthesameamountoftimeindependentof
thequantityofmodificationsmade.
Rollback
cantake
longer
,dependinguponthe
quantityofdataandcharacterof
modifications,
whichweremadewithinthedatabase.
Tableoperations.Updates
Updatekeyoperationisthesameasinsertoperation
wehaveupdatedkey2andsetupnewvalue2.
Tableoperations.Bulkoperations
Ifyouaregoingtoinsertorupdateabigdatasetthenfirstexecuteinsert,update,remove
commandasmanytimesasyouneedandthencalltran.Commit()
Callingtran.
Commit
aftereveryoperation,
willnotmaketablephysicalfilebigger
butwill
takemoretimethenoneCommitafteralloperations.
//THISISFASTER
for(inti=0i<1000000i++)
{
tran.Insert<int,int>(t1,i,i)
}
tran.Commit()
//THISISSLOWER

for(inti=0i<1000000i++)
{
tran.Commit()
}
Tableoperations.Randomkeyswhilebulkinsert.
Dbreezealgorithmsarebuilttoworkwithmaximumefficiencywhileinsertinginbulksorted
ascendingdata.
for(inti=0i<1000000i++)
{
}
tran.Commit()
//or
DateTimedt=DateTime.Now
for(inti=0i<1000000i++)
{
tran.Insert<DateTime,int>(t1,dt,i)
dt=dt.AddSeconds(7)
}
tran.Commit()
Theabovecodewillexecute9seconds(year2012and1.5secondsinyear2015).
Ifyoustarttoinsertdatainrandomorderitcantakeuptosomeminutes.Thatswhy,ifyou
haveinmemorybigdataset,beforesavingittothedatabase,sortitascendinginmemory
bykeyandtheninsert.Itwillspeedupyourprogram.
IfyoumakecopyfromotherdatabasestoDBreeze,takeachunk(e.g.1MLNrecords),sort
itinmemorybykeyascending,insertintoDBreeze,themtakeanotherchunkandsoon.
Tableoperations.PartialInsertorUpdate
InDBreeze
maximalkeylength
inbytesis65535(UInt16.MaxValue)andmaximal
value
length
is2147483647(Int32.MaxValue).
Itsnotpossibletosaveasavaluebytearraybiggerthan2GB.Forbiggerdataelementswe
willhavetodevelopinthefutureotherstrategy.
InDbreezewehaveabilityofapartialvalueupdateorinsert.Itspossiblebecausevalues
arestoredasbyte[].Itdoesntmatterwhichdatatypeisstoredalreadyinthetableyoucan
alwaysaccessitandchangeasbytearray.
DBreezehasspecialnamespaceinside,whichallowsyoueasilytoworkwithbytearrays.
usingDBreeze.Utils
Nowyoucanconvertanystandarddatatypeintobytearrayandback.
Wewillachievethesameeffectinallfollowingrecords:
tran.Insert<int,
int
>("t1",10,1)
//or
tran.Insert<int,
byte[]
>("t1",10,((int)1).To_4_bytes_array_BigEndian())
//or
tran.Insert<int,
byte[]
>("t1",10,newbyte[]{0x80,0x00,0x00,0x01})
Aboveinstructionscanberunonebyoneandwillbringtotheresultthenunderkey10we
willhavevalue1.
Andthesameresultwewillachievehavingrun4followinginstructions:
tran.InsertPart<int,byte[]>("t1",10,newbyte[]{0x80},0)
//orthesame
tran.InsertPart<int,byte[]>("t1",10,newbyte[]{0x80,0x00},0)
tran.InsertPart<int,byte[]>("t1",10,newbyte[]{0x00,0x01},2)
Thefourthparameteroftran.InsertPartisexactlytheindexfromwhichwewanttoinsertour
byte[]array.
This
technique
canbeusedbyus,ifwethinkaboutthevalueasaboutthe
setofcolumns
oftheknownlength,likein
standardSQLdatabases
,andgivesusabilitytochangeevery
columnseparately,withoutchangingallothervalues.
Note,
youcanalways
switchtobyte[]datatype
invaluesandinkeys
tran.Insert<int,
int
>
//or
tran.Insert<int,
byte[]
>
ifitsinterestingforyou.
Note,
Ifyouwanttoinsertorupdatevaluestartingfromthe
indexwhichisbiggerthen
currentvalue
length,theemptyspacewillbefilledwithbyte[]{0}.
Wedidnthavebeforekey12andnowweareexecutingfollowingcommands:
tran.InsertPart<int,byte[]>("t1",12,newbyte[]{0x80},
0
)
tran.InsertPart<int,byte[]>("t1",12,newbyte[]{0x80},
10
)
Valueasbyte[]willlooklikethis:
"0x800x000x000x000x000x000x000x000x000x000x80"
Note
,Dbreezewilltrytousethe
samephysicalfilespacewhilerecordupdate,
ifexisting
recordlengthissuitableforthis.
Tableoperations.DataFetching.Select.
Methodtran.Selectisdesignedforgettingonesinglekey:
tran.Commit()
varrow=tran.Select<int,
int
>("t1",10)
//
orwillworkalsogood
varrow=tran.Select<int,
byte[]
>("t1",10)
Afterselectyoumustsupplyingenericformatdatatypesforthekeyandvalue.
Inourcase,wewanttoreadfromtablet1keyoftypeint(itsvalue10).
SelectalwaysreturnsavalueoftypeDBreeze.DataTypes.Row
.
Wecan
starttovisualize
thekeyvalue
onlyafterchecking,
iftablehassuchvalueinside.
Row
has
propertyExists
:
{
tran.Commit()
varrow=tran.Select<int,int>("t1",10)
byte[]btRes=null
intres=0
intkey=0
if(
row.Exists
)
key=row.Key
res=row.Value
//btReswillbenull,becausewehaveonly4bytes
btRes=row.GetValuePart(12)
//btReswillbenull,becausewehaveonly4bytes
btRes=row.GetValuePart(12,1)
//willreturn4bytes
btRes=row.GetValuePart(0)
//willreturn4bytes
btRes=row.GetValuePart(0,4)
}
So,ifrowexists,wecanstarttofetchitskey(
row.Key
),fullrecord
row.Value
(itwillbe
automaticallyconvertedfrombyte[]tothedatatype,whichyougavewhileformingSelect).
Andindependentfromtherecorddatatype,Rowhasmethod
GetValuePart
withoverloads
whichwillhelpyoutogetvaluepartiallyandalwaysasbyte[].DBreeze.Utilsextensionscan
helptoconvertvaluestootherdatatypes.
Ifwehadinthevalue,startingfromindex4storedsomekindofulong,whichresides8
bytes,wecansay:
ulongx=row.GetValuePart(4,8).To_UInt64_BigEndian()
Note
,that
DBreeze.UtilsconversionalgorithmsareexactlysharpenedforDBreeze
datatypes,becausetheycreatesortablebyte[]sequencesincomparewith.NETbuilt
inbyte[]conversionfunctions
.
Tableoperations.DataFetching.NULL
tran.Insert<int,int?>(t1,10,null)
varrow=tran.Select<int,int?>(t1,10)
if(row.Exists)
{
int?val=row.Value//valwillbenull
}
Tableoperations.DataFetching.Orderby.OrderbyDescending.
When
Dbreezestoresdata
inthetableits
automatically
storedinthe
sorted
order.Thats
whyallrangeselectsareveryfast.Thisexampleistakenfromsatelliteprojectintegrated
intoDBreezesolution,whichiscalledVisualTesterfromclassDocuExamples:
publicvoidExample_FetchingRange()
{
engine.Scheme.DeleteTable("t1")
DBreeze.Diagnostic.SpeedStatistic.StartCounter("INSERT")
DateTimedt=DateTime.Now
for(inti=0i<1000000i++)
tran.Insert<DateTime,byte?>("t1",dt,null)
dt=dt.AddSeconds(7)
tran.Commit()
DBreeze.Diagnostic.SpeedStatistic.StopCounter("INSERT")
DBreeze.Diagnostic.SpeedStatistic.PrintOut(true)
DBreeze.Diagnostic.SpeedStatistic.StartCounter("FETCH")
foreach(varrowintran.SelectForward<DateTime,byte?>("t1"))
//Console.WriteLine("K:{0}V:{1}",
row.Key.ToString("dd.MM.yyyyHH:mm:ss"),(row.Value==null)?"NULL":
row.Value.ToString())
DBreeze.Diagnostic.SpeedStatistic.StopCounter("FETCH")
DBreeze.Diagnostic.SpeedStatistic.PrintOut(true)
Asmallbenchmarkforthisprocedure:
INSERT:10361ms28312951ticks
FETCH:4700ms12844468ticks
AllrangeselectsmethodsinDBreezereturnIEnumerable<Row<TKey,TValue>>,so
theycanbeusedinforeachstatements.
Ifyouwant,youcanbreakfromforeachinanymoment.
Tolimitthequantityofthedatayoucanuse,eitherbreakiterationoruse
Take
statement
:
foreach(varrowintran.SelectForward<DateTime,byte?>("t1").
Take
(100))
SelectForward
startsfromthefirstkeyanditeratesforwardtothelastkeyinsorted
ascending
order.
SelectBackward
startsfromthelastkeyanditeratesbackwardtothefirstkeyinsorted
descending
order.
Transactionhasmoreselfexplainedmethods:
IEnumerable<Row<TKey,TValue>>
SelectForwardStartFrom
<TKey,TValue>(string
tableName,TKeykey,boolincludeStartFromKey)
Note,ifkeyisnotfoundthenitstarts
fromthenextavailablekeyforwardinascendingorder,ideaofnonexistingsupplied
parameterconcernsalliterationalmethods.
SelectBackwardStartFrom
<TKey,TValue>(stringtableName,TKeykey,bool
includeStartFromKey)iteratesfromthegivenkeydownindescendingorder.
SelectForwardFromTo
<TKey,TValue>(stringtableName,TKeystartKey,bool
includeStartKey,TKeystopKey,boolincludeStopKey)
SelectBackwardFromTo
<TKey,TValue>(stringtableName,TKeystartKey,bool
includeStartKey,TKeystopKey,boolincludeStopKey)
DONT
USE
LINQ
after
SelectForwardorSelectBackwardforfilteringlikethis:
tran.SelectForward<int,int>(t1).Where(r=>r.Key>10).Take(10)

Becauseitwillwork
muchmuchmuchslower
thenspeciallysharpenedmethods,
useinstead:
tran.
SelectForwardStartFrom
<int,int>("t1",10,false).Take(10)
Andfinallytwomorespecialmethods:
SelectForwardStartsWith
<TKey,TValue>(stringtableName,TKeystartWithKeyPart)
and
SelectBackwardStartsWith
<TKey,TValue>(stringtableName,TKeystartWithKeyPart)
Yourememberthatalldatatypeswillbeconvertedintobyte[].
Soifintablewehavekeys
byte[]{0x12,0x15,0x17}
byte[]{0x12,0x16,0x17}
byte[]{0x12,0x15,0x19}
byte[]{0x12,0x17,0x18}
then
SelectForwardStartsWith<byte[],int>(t1,newbyte[]{0x12})
willreturnusallkeys
SelectForwardStartsWith<byte[],int>(t1,newbyte[]{0x12,0x15})
willreturnusonly2keys
byte[]{0x12,0x15,0x17}
byte[]{0x12,0x15,0x19}
<byte[],int>(t1,newbyte[]{0x12,0x15})
willreturnusonly2keysindescendingorder
byte[]{0x12,0x15,0x19}
byte[]{0x12,0x15,0x17}
willreturnus1key
byte[]{0x12,0x17,0x18}
and
willreturnnothing.
Havingthisideawecaneffectivelyworkwithstrings:
tran.Insert<string,string>(t1,w,w)
tran.Insert<string,string>(t1,ww,ww)
tran.Insert<string,string>(t1,www,www)
then
SelectForwardStartsWith
<string,string>(t1,ww)
willreturnus
ww
www
and
<string,string>(t1,ww)
willreturnus
www
ww
Tableoperations.Skip
InDbreezewehaveabilitytostartiterationsafterSkippingsomeotherkeys:
thiscommandskipsskippingQuantityelementsandthenstartsenumerationinascending
order:
SelectForwardSkip
<TKey,TValue>(stringtableName,ulongskippingQuantity)
thiscommandskipsskippingQuantityelementsbackwardandthenstartsenumerationin
descendingorder:
IEnumerable<Row<TKey,TValue>>
SelectBackwardSkip
<TKey,TValue>(string
tableName,ulongskippingQuantity)
thiscommandskipsskippingQuantityelementsfromthespecifiedkey(ifkeyisnotfound
thennextoneafteritwillbetakenasskipped1)andthenstartsenumerationinascending
order:
SelectForwardSkipFrom
<TKey,TValue>(stringtableName,TKeykey,ulong
skippingQuantity)
thiscommandskipsskippingQuantityelementsbackwardfromthespecifiedkeyandthen
startsenumerationindescendingorder:
SelectBackwardSkipFrom
<TKey,TValue>(stringtableName,TKeykey,ulong
skippingQuantity)
Note,thatskipneedstoiterateviakeys,tocalculateexactskippingquantity
.Thats
whydeveloperhasalwaystotakeintoconsiderationtheideaofthefindingcompromise
betweenspeedandskippingquantity.Skipping1MLN,ofelementsin
anydirection
starting
from
anykey
willtake4secondswithInteli78coresandSCSIdrive8GBRAM(year2012).
Skipof100000recordswilltake400ms,10000willtake40msrespectively.
So,ifyouaregoingtoimplementgridpaging,thenjustrememberfirstshowninthegridkey
andthenskipfromitquantityofshowninthegridelementsusing
SelectForwardSkipFrom
or
SelectBackwardSkipFrom
.
Tableoperations.Count.
Forgetting
Table
records
quantity
use:
ulongcnt=tran.
Count
("t1")
Countiscalculatedwhileinsertingandremovingoperationsandisalwaysavailable.
Tableoperations.Max.
varrow=tran.
Max
<int,int>("t1")
if(row.Exists)
{
//etc...
}
Tableoperations.Min.

varrow=tran.
Min
<int,int>("t1")
if(row.Exists)
{
//etc...
}
Tableoperations.Readingfromnonexistingtable
Ifyoutrytoreadfromnonexistingtable,this
tablewillnobecreated
inthefilesystem.
tran.Countwillreturn0
tran.Select,tran.Min,tran.Maxwillreturnrowwithrow.Exists==false
Rangeselectsliketran.SelectForwardetc.willreturnnothinginyourforeachstatement.
Tableoperations.Removingkeys
To
removeonekey
use
tran.RemoveKey<int>("t1",10)
tran.Commit()
To
Removeallkeys
use
tran.RemoveAllKeys
(stringtableName,boolwithFileRecreation)
Note
,if
withFileRecreation
parameterissetto
true
,thenwe
dont
needto
Commit
this
modification,itwillbedoneautomatically.Thefilewhoholdsthetablewillberecreated.
Note
,if
withFileRecreation
parameterissetto
false,
theolddatawillbenotvisibleany
more,buttheoldinformationwillstillresideinthetable.We
needCommit
afterthis
modification.
Tableoperations.Changekey
Wehaveanabilitytochangethekey.
Afterthesecommands:
tran.Insert<int,int>(t1,10,10)
tran.ChangeKey<int>("t1",10,11)
tran.Commit()

wewillhaveinthetableonekey11withthevalue10.
Afterthesecommands:
tran.ChangeKey<int>("t1",10,11)
tran.Commit()
wewillhaveinthetableonekey11withthevalue10.(oldvalueforthekey11willbelost)
Storingobjectsinthedatabase
Forstoringobjectsinthetablewehave3extradatatypeswhichareaccessiblevia
DBreeze.DataTypesnamespace.
DbXML<T>
willautomaticallyusebuiltin.NETXMLserializeranddeserializerforobjects.
Slowerthenothersinbothoperationsfurthermoredataresidesmuchmorephysicalspace,
thenothers.
DbMJSON<T>
MicrosoftJSON,willautomaticallyusebuiltin.NETJSON
(System.Web.Script.Serialization.JavaScriptSerializer)serializeranddeserializerfor
objects.MuchbetterthenXMLbutnotsogoodasserializerprovidedby
http://json.codeplex.com/
JSON.NET,thoughresidesapproximatelythesamephysical
spaceonthedisk.
DbCustomSerializer<T>
givesyouabilitytoattachyourownserializerlike
http://json.codeplex.com/
.
ToattachJSON.NET,downloadit,refertoyourprojectandfillsomelines:
DBreeze.Utils.CustomSerializator.Serializator=JsonConvert.SerializeObject
DBreeze.Utils.CustomSerializator.Deserializator=JsonConvert.DeserializeObject
nowyoucanuseserializationanddeserializationprovidedbyJSON.NET.
ButifyoudontwanttouseJSON.NET,tryMicrosoftJSON.Itsabout40%sloweron
deserializationand510%sloweronserializationthenJSON.NET.
Usealloftheminfollowingmanner:
publicclassArticle
{

publicuintId{getset}
publicstringName{getset}
}
publicvoidExample_InsertingObject()
{
engine.Schema.DeleteTable("Articles")
tran.SynchronizeTables("Articles")
uintidentity=0
varrow=tran.Max<uint,byte[]>("Articles")
if(row.Exists)
identity=row.Key
identity++
Articleart=newArticle()
Id=identity,
Name="PC"
tran.Insert<uint,DbMJSON<Article>>("Articles",identity,art)
tran.Commit()
}
}
Note
,DbMJSON,DbXML,DbMJSON,DbCustomSerializerhaveoverloadedoperatorand
youcanspecifyartwithoutsayingnewDbMJSON<Article>,justsayart:
tran.Insert<uint,DbMJSON<Article>>("Articles",identity,
art
)
//or
tran.Insert<uint,DbXML<Article>>("Articles",identity,
art
)
//or
tran.Insert<uint,DbCustomSerializer<Article>>("Articles",identity,
art
)
Gettingobjects:
foreach(varrowintran.SelectForward<uint,DbMJSON<Article>>("Articles").Take(10))
{
//Note
row.Value
willreturnusDbMJSON<Article>
//row.Value
//ButweneedArticle
//
Article
a=row.Value.
Get
//Oritsserializedrepresentation
//
string
aSerialized=row.Value.
SerializedObject
Multithreading
InDbreezetablesarealwaysaccessibleforparallelREADoflastcommitteddatafrom
multiplethreads.
Note
,while
onethread
is
writingdata
intothetable,
otherthreads
will
not
be
able
to
write
datainthesametable(tablelock),tillwritingthreadreleasesitstransaction,theywill
waitinaqueue.
Note
,while
onethread
is
writingdata
intotable,
otherthreads
can
inparallel
read
already
committeddata
.
Note
,if
one
of
threads
needs,insideofthetransaction,to
read
datafromthetablesandit
wantstobesurethattilltheendoftransaction
otherthreads
will
notmodify
thedata,this
threadmust
reservetablesforsynchronizedread
.
{
tran.SynchronizeTables("table1","table2")
Transactionalsohasmethodfortablessynchronization.
tran.SynchronizeTables
Thismethodhasoverloadsandyoucansupplyasparameters:List<string>orparams
string[].
SynchronizeTablescanberunonlyonceinsideofthetransaction.
All
reads
canbedividedon
twocategories
byusagetype:
Readforreporting
Readformodification
Basedonthisideathewholemultithreadedlayerisbuilt.
Multithreading.Readforreporting
Ifyouthinkthatthereisnonecessitytoblocktable(s)andotherthreadscouldwritedatain
paralleljustdontusetran.SynchronizeTables.
Thistechniqueisapplicableinallreportingcases.Ifuserneedstoknowhisbankaccount
state,wedontneedtoblockthetablewithaccountinformation,justreadaccountstateand
returnit.Doesntmatterthatinthismomenthisaccountstateischangingitsaquestionof
amoment.Ifuserrequestshisaccountstatein5minuteshewillgetalreadymodified
account.
Therearesomethingswhichmustbeunderstood.
ForexamplewemakeiterationviatableItems,becausesomeonehasrequesteditsfulllist.
Letsassumethatthereare100items
List<Item>items=newList<Item>()
foreach(varrowintran.SelectForward<ulong,DbMJSON<Item>>(Items))
{
items.Add(row.Value.Get)
//wehaveiteratedover50itemsandinthismomentotherthreaddeleteditemId1
andcommittedtransaction
//Result:itsaquestionofthemomentthisitemwillbeaddedtothefinalList,it
doesntmatterinthiscase.
//wehaveiteratedalready75itemsandinthismomentotherthreaddeleteditemId
90andcommittedtransaction
//after89wewillgetitem91
//Result:itsaquestionofthemoment,item90willnotbeaddedtothefinalList,it
doesntmatterinthiscase.
AndifyouwanttobesurethatotherthreadswillnotmodifyItemstable,whileyouare
fetchingthedata,use
tran.SynchronizeTables(Items)
Ifyoutakearowfromatablealwayscheckifitexists.
Ifyourdataprojectionisspreadamongmanytables,firstgetallpiecesofthedata
fromdifferenttables,alwayscheckingifrow.Exists,incaseofdirectselects,andonly
whenyouhavefullobjectconstructedthenreturnittothefinalprojectionasaready
element.
Noteifyouhavereceivedrowanditexists.Itdoesntmeanthatyouhavealready
acquiredthevalue.Valuewillbereadonlywhenyouchoosepropertyrow.Value(lazy
valueloading).Ifotherthreadremovesvalueinbetween,afteryouhaveacquiredthe
row,butstilldidntacquiredvalue,thenvaluewillbereturnedinanycase,because
afterremovingdatastillstaysonthedisk,onlykeysaremarkedasdeleted.Andthis
behaviourfornotsynchronizedreadshouldbeok,becauseitsaquestionofthe
moment.
Ifyouhaveacquiredrowanditexists,inonethread,nowyouaregoingtogetthe
value,butinthismomentotherthreadupdatesvalue,thenyouthreadwillreceive
updatedvalue.
IncaseifyourthreadisgoingtoretrievevalueandinthismomentDBreeze.Scheme
deletestabletheninsideoftransactionexceptionwillberaised,controlledby
trycatchintegratedintousingstatement.
Thesamewillhappenifotherthreadexecutestran.RemoveAllKeys(yourreading
table,truewithFileRecreation).Yourreadingthreadwillgetexceptioninsideofthe
transaction.Butallwillbeokifotherthreadsremovesdatawithoutfilerecreation,if
tran.RemoveAllKeys(yourreadingtable,falsewithFileRecreation).
YoumustuseScheme.DeleteTable,Scheme.RenameTableandtran.RemoveAllKeys
withtablerecreationsemantically.
Eitherinconstructor,afterengineinitialization,orfortemporarytables,whichareusedfor
subcomputationwiththehelpofdatabase,anddefinitelyonlybyonethread.Fortables
whichareunderreadwritepressure,bettertousetran.RemoveAll(false)andthenoneday
tocompactthistablebycopyingexistingvaluesintonewtable,andrenamingnewtableto
oldtable.
Tablescopying/compaction
Copyingofthedatabettertomakeonbyte[]level,itwillbefasterthentocastandserialize/
deserializeobjects.
IfyouhadtableArticles<ulong,DbMJSON<Atricle>>
Copyitlikethis:
foreach(varrowintran.SelectForward(<byte[],byte[]>(Articles)))
{
tran.Insert<byte[],byte[]>(ArticlesCopy,row.Key,row.Value)
}
tran.Commit()
thenyoucanrenameoldtableScheme.RenameTable(ArticlesCopy,Articles)
andgoontoworkwithArticletable
foreach(varrowintran.SelectForward(<long,DbMJSON<Atricle>>(Articles)))
{
...
}
Note
,wecreateforeachloopwhichreadsfromonetableandafterthatwritesintotheother
table.FromHDDpointovviewwemakesuchoperation:
RWRWRWRW..
IfyouhavemechanicalHDD,itsheadmustalwaysmovebetweentwofilestocompletethis
operation,whatisnotsoefficient.
Toincreaseperformanceofthecopyprocedureweneedfollowingsequence:
RRRRWRRRRWRRRRW.
So,firstwereadtothememoryabigchunk(1K/10K/100K/1MLNofrecords)andthensortit
bykeyinascendingorderandinsertitinbulktothecopytable.
Dictionary<TKey,TValue>willnotbeabletosortbyte[].Forthisweneedtoconstruct
hashstringusingDBreeze.Utils:
byte[]bt=newbyte[]{0x08,0x09}
stringhash=bt.ToBytesString()
thenputthishashakeyforDictionary.Copyprocedurewith
RRRRWRRRRWRRRRW.
sequence:
usingDBreeze.Utils
inti=0
intchunkSize=100000
Dictionary<string,KeyValuePair<byte[],byte[]>>cacheDict=new
Dictionary<string,KeyValuePair<byte[],byte[]>>()
{
cacheDict.Add(
row.Key.ToBytesString()
,newKeyValuePair<byte[],byte[]>
(
row.Key,
row.Value
)
)
i++
if(i==chunkSize)
{
//savingsortedvaluestothenewtableinbulk
foreach(varkvpincacheDict.OrderBy(r=>r.Key))
{
tran.Insert<byte[],byte[]>(ArticlesCopy,kvp.Value.Key,
kvp.Value.Value)
}
cacheDict.Clear()
i=0
}
}
//Ifsomethingleftincacheflushit
foreach(varkvpincacheDict.OrderBy(r=>r.Key))
{
tran.Insert<byte[],byte[]>(ArticlesCopy,kvp.Value.Key,kvp.Value.Value)
}
cacheDict.Clear()
tran.Commit()
Note
,actuallywedontneedtosortdictionary,becauseSelectForwardfromtableArticles
givesusalreadysortedvaluesandinsortedsequencetheywillmigrateinto
cacheDictionary,soourcompletecodewilllooklikethis:
inti=0
intchunkSize=100000
Dictionary<byte[],byte[]>cacheDict=newDictionary<byte[],byte[]>()
{
cacheDict.Add(row.Key,row.Value)
i++
if(i==chunkSize)
{
//savingsortedvaluestothenewtableinbulk
foreach(varkvpincacheDict)
{
tran.Insert<byte[],byte[]>(ArticlesCopy,kvp.Key,kvp.Value)
}
cacheDict.Clear()
i=0
}
}
//Ifsomethingleftincacheflushit
foreach(varkvpincacheDict)
{
tran.Insert<byte[],byte[]>(ArticlesCopy,kvp.Key,kvp.Value)
}
cacheDict.Clear()
tran.Commit()
Multithreading.Readformodification
Thistechniqueisusedwhenyouneedtogetdata(select)beforemodification(insertor
updateetc.):
privateboolAddMoneyOnAccount(uintuserId,decimalsum)
{
{

try
stringtableUserInfo=UserInfo+userId
tran.SynchronizeTables(tableUserInfo)
//afterSynchronizeTables,besurethatnoneoftheotherthreadswillwritein
tabletableUserInfo,tillthetransactionwillbereleased.
//nowwereadthestateoftheuseraccount
varrow=tran.Select<string,decimal>(tableUserInfo,Account)
decimalaccountState=0
if(row.Exists)
accountState=row.Value
//nowwechangethesumoftheusersaccount
accountState+=sum
tran.Insert<string,decimal>(tableUserInfo,Account,accountState)
tran.Commit()
catch(Exceptionex)
Console.WriteLine(ex.ToString())
returnfalse
}
}
returntrue
}
TableWRITE,ResolvingDeadlockSituation
Ifwewriteonlyinonetableinsideoftransactionandforothertablesuseunsynchronized
read,wedontneedtouse
SynchronizeTables
{

}
Butwhenwehaveinserted/updated/Removedakeyinthetable,DBreezewillautomatically
blockthewholetableforWrite,likeSynchronizeTables(t1)wouldbeused,tilltheendofthe
transaction.
Infollowingexample,transactionfirstblockstablet1andthent2
{
Imagine,thewehaveparallelthreadwhichwritesinthesametablesbutinothersequence:
{
Thread2hasblockedtablet2,whichisgoingtobereadbyThread1,andThread1has
blockedtablet1,whichisgoingtobereadbyThread2.
Suchsituationiscalleddeadlock.
DbreezeautomaticallydropsoneofthesethreadswithDeadlockException,andtheother
threadwillbeablesuccessfullyfinishitsjob.
Butthisisonlyapartofthesolution.Tomaketheprogramdeadlocksafeuseinboth
threads
SynchronizeTables
construction:
Thread1:
{
tran.SynchronizeTables(t1,t2)
Thread2:
{
tran.SynchronizeTables(t1,t2)
Boththreadswillbeexecutedwithoutexceptions,onebyoneabsolutedefencefrom
thedeadlocksituation.
TableWRITE,READorSYNCHROREAD,Datavisibilityscope
Inthefollowingexamplewereadarowfromtablet1.
{
varrow=tran.Select<int,int>(t1,1)
}
Wedidntusetran.SynchronizeTablesconstructionandwedidntwritetothistablebefore,
sowewillseeonlylastcommitteddata,evenifotherthreadischangingthesamedatain
parallel,thistransactionwillreceiveonlylastcommitteddataforthistable.
Buteverythingchangeswhentransactionhasatableinmodificationlist:
{
//Tablet1isinmodificationlistofthistransactionandallreadsfrom
thistableautomaticallyreturnactualdata,evenbeforecommit

//thisrow.Valuewillreturn157
}
Allreadsofthetable(onlyinsidecurrenttransaction),ifitsinmodificationlist(by
SynchronizeTablesorjustinsert/update/remove)willreturnmodifiedvaluesevenifthedata
wasnotcommittedyet:
{
tran.Commit()
}
{
//row.Valuewillreturn99likeotherparallelthreadswhichread
thistable
//butthisthreadwantsalsotomodifythistable
//row.Valuewillreturn117(otherthreadswillsee99)
tran.RemoveKey(t1,1)
//row.Existswillbefalse(otherthreadswillsee99)
/
tran.Commit()
}
TableSynchronizationbyPATTERN
BecauseinNoSqlconceptwehavetohavedealswithmanytablesinsideofone
transaction,DBreezehasspecialconstructionsfortableslocking.Alltheseconstructionsare
availableviatran.SynchronizeTables
.
Again,tran.SynchronizeTablescanbeusedonlyonceinsideofanytransactionbeforeany
modificationcommand,butcanbeusedafterreadcommands:
ALLOWED:
{
tran.SynchronizeTable(t1)
tran.Commit()
}
{
tran.SynchronizeTable(t1,t2)
tran.Commit()
}
{
List<string>ids=newList<string>()
foreach(varrowintran.SelectForward<int,int>(Items))
{
ids.Add(Article+row.Value.ToString())
}
tran.SynchronizeTable(ids)
tran.Commit()
}
Note
,itspossibletoinsertdataintotableswhichwerenotsynchronizedby
SynchronizeTable
{
tran.Insert<int,int>(
t2
,1,99)
tran.Commit()
}
Butthisisbettertousefotemporarytables,foravoidingdeadlocks.Toadduniquenessto
thetablename(temporarytablename)addThreadId:
{
try{
string
tempTable
=temp+tran.ManagedThreadId+_more
//
incaseifpreviousprocesswasinterruptedandtempTablewasnot
deleted
engine.Scheme.DeleteTable(
tempTable
)
tran.Insert<int,int>(
tempTable
,1,99)
//dooperationswithtemptable.....
tempTable
)
tran.Commit()
}catch(System.Exceptionex)
{
//exhandle
tempTable
)
}
}
NOTALLOWED:
{
tran.Commit()
}
Tosynchronizetablesbypatternweusespecialsymbols:
*allothersymbols
#allothersymbolsexceptslash,followedbyslashandanyothercharacter
$allothersymbols,exceptsslash
tran.SynchronizeTable(Articles
*
)willmeanthatweblockforwritingalltableswhichstart
fromthewordArticles,like:
Articles123
Articles231
etc.
Articles123/SubItems123/SubItems123
andsoon.
#
/Items
*
)willmeanthatweblockforwritingfollowing
tables,like:
Articles123/Items1257/IOo4564
butwedontblock
Articles123/SubItems546
$
)willmeanthatweblockforwritingfollowingtables,like:
Articles123
Articles456
andwedontblock
Articles456/Items...
Slashcanbeeffectivelyusedforcreatinggroups.
Surewecancombinepatternsinonetran.SynchronizeTablecommand:
tran.SynchronizeTable(Articles1/Items$,Articles#/SubItems*,
Price1,Price#/Categories#/El*)
NonUniqueKeys

InDBreezetablesallkeysmustunique.
Buttherearealotofmethodshowtostorenonuniquekeys.
Oneofthemisforeverynonuniquekeycreateaseparatetableandstoreallreferenceto
thiskeyinside.Sometimesthisapproachisgood.
Butthereisanotherusefulapproach.
Note,thatDBreezeisaprofessionaldatabaseforhighperformanceandmissioncritical
applications.DeveloperspendsalittlebitmoretimefortheDataAccessLayer,butgets
backveryfastresponsesfromdatabase.
ImaginethatyouhaveaplentyofArticlesandeveryofithaspriceinside.Youknowthat
oneoftherequirementsofyourapplicationistoshowarticlessortedbyprice.Another
requirementistoshowarticlesinpricerange.
Itcanmeanthatexceptthetablewhoholdsarticlesyouwillneedaspecialtablewhereyou
willstorepricesaskeys,tobeabletouseDBreezeSelectForwardStartFromor
SelectForwardFromTo.
Developer,whileinsertingonearticle,hastofilltwotables(itsaminimumforthisexample)
ArticlesandPrices.
Buthowwecanstorepricesaskeytheyarenotunique.
Thenwewillmakethemunique.
usingDBreeze
usingDBreeze.Utils
usingDBreeze.DataTypes
publicclassArticle
{
publicArticle()
Id=0
Name=String.Empty
Price=0f
publicuintId{getset}
publicfloatPrice{getset}
}

publicvoidExample_NonUniqueKey()
{
engine.Schema.DeleteTable("Articles")
uintid=0
Articleart=newArticle()
Name="Notebook",
Price=100.0f
id++
tran.Insert<uint,DbMJSON<Article>>("Articles",id,art)
byte[]idAsByte=id.To_4_bytes_array_BigEndian()
byte[]priceKey=art.Price.To_4_bytes_array_BigEndian().Concat(idAsByte)
Console.WriteLine("{0}Id:{1}IdByte[]:{2}btPriceKey:{3}",art.Name,id,
idAsByte.ToBytesString(""),priceKey.ToBytesString(""))
tran.Insert<byte[],byte[]>("Prices",priceKey,null)
art=newArticle()
Name="Keyboard",
Price=10.0f
id++
idAsByte=id.To_4_bytes_array_BigEndian()
priceKey=art.Price.To_4_bytes_array_BigEndian().Concat(idAsByte)
art=newArticle()
Name="Mouse",
Price=10.0f
id++
art=newArticle()
Name="Monitor",
Price=200.0f
id++
//thisarticlewasaddedlaterandnotreflectedinthepostexplanation
art=newArticle()
Name="MousePad",
Price=3.0f
id++
tran.Commit()
Console.WriteLine("***********************************************")
//Fetchingdata>=
//WeareinterestehereinArticleswiththecost>=10
floatprice=10f
uintfakeId=0
byte[]searchKey=
price.To_4_bytes_array_BigEndian().Concat(fakeId.To_4_bytes_array_BigEndian())
Articleart=null
foreach(varrowintran.SelectForwardStartFrom<byte[],byte[]>("Prices",
searchKey,true))
Console.WriteLine("Foundkey:{0}",row.Key.ToBytesString(""))
varartRow=tran.Select<uint,DbMJSON<Article>>("Articles",
row.Key.Substring(4,4).To_UInt32_BigEndian())
if(artRow.Exists)
art=artRow.Value.Get
Console.WriteLine("Articel:{0}Price:{1}",art.Name,art.Price)
Console.WriteLine("***********************************************")
//Fetchingdata>
//WeareinterestehereinArticleswiththecost>10
floatprice=10f
uintfakeId=UInt32.MaxValue
byte[]searchKey=
price.To_4_bytes_array_BigEndian().Concat(fakeId.To_4_bytes_array_BigEndian())
Articleart=null

foreach(varrowintran.SelectForwardStartFrom<byte[],byte[]>("Prices",
searchKey,true))
Console.WriteLine("Foundkey:{0}",row.Key.ToBytesString(""))
varartRow=tran.Select<uint,DbMJSON<Article>>("Articles",
row.Key.Substring(4,4).To_UInt32_BigEndian())
if(artRow.Exists)
art=artRow.Value.Get
Console.WriteLine("Articel:{0}Price:{1}",art.Name,art.Price)
EveryarticlewhenisinsertedtoArticlestablereceivesitsuniqueidottypeuint:
Articles<uint,DbMJSON<Article>>(Articles)
YourememberthatinnamespaceDBreeze.Utilstherearealotofextensionforconverting
differentdatatypestobyte[]andback.Wecanconvertdecimals,doubles,floats,integers
etc.tobyte[]andback.
Articlepriceisfloatinourexampleandcanbeconvertedtobyte[4](sortablebytearrayfrom
DBreeze.Utils,System.BitConverterwillnotgiveyousuchresults).
Asyouseewehad4articles2ofthemhadthesameprice.
Weachieveuniquenessofthepriceonthebytelevelbyconcatenatingtwobytearray.
Firstpartisapriceconvertedtobytearray(forArticle
Keyboard
):
float10.0f>AE0F4240
SecondpartisuintIdfromtableArticlesconvertedtobytearray(forArticle
Keyboard
):
uint2>00000002
whenweconcatenatebothbytearraysforeveryarticlewewillhavesuchresult:
NotebookId:1btPriceKey:AF0F424000000001
//100f
KeyboardId:2btPriceKey:AE0F424000000002
//10f
MouseId:3btPriceKey:AE0F424000000003
//10f
MonitorId:4btPriceKey:AF1E848000000004
//200f
Thatsallexactlythesefinalbytearraysweinsertintotableprices.
Nowfetchingdata
SelectForwardandBackwardfromtablePriceswillgiveyoualreadysortedbypriceresults.
MoreinterestingistogetAllpricesstartingfrom10f.
Forthiswewillusetran.SelectForwardStartFrom(Prices,btKey,true)
weneedtogetbtKey.
Wetakeourdesirable10fandconverttobyte[]
floatfindPrice=10f
byte[]btKey=findPrice.To_4_bytes_array_BigEndian()
thenweneedtoconcatenatewiththebtKeyfullarticleidandhereisatrick:
uintid=0
btKey=btKey.Concat(id.To_4_bytes_array_BigEndian())
willgiveussuchbtKey:
AE0F424000000000
ifweuseitintran.SelectForwardStartFrom(Prices,btKey,true)
wewillreceiveallprices>=10f.
Ifwe
uintid=UInt32.MaxValue
btKey=btKey.Concat(id.To_4_bytes_array_BigEndian())
willgiveussuchbtKey:
AE0F4240FFFFFFFF
applyingsuchkeyintran.SelectForwardStartFrom(Prices,btKey,true)
wewillreceivepriceonly>10f.
Surewhenyougotthekeyfromvalueprice(itsbyte[]),youcanmake
row.Value.Substring(4,4).To_UInt32_BigEndian()receiveyouuintidfromtableArticlesand
retrievevaluefromtableArticlesbythiskey.
[20120521]
Fractaltablesstructure.
Wecallitwithafancywordfractal,becauseithasselfsimilarstructure.
Actually,itsanabilitytostoreinanykindofavalue(ofaKey/Valuetable)from1toNother
tables+extradata.Andinanykindofanestedtablekeysvaluesotherfrom1toNtables+
extradataandsoon,tillyouresourcesletyoudothat.Suchmultidimensionalstorage
concept.
Itcanalsomeanthatinonevaluewecanstoreobjectofanycomplexitykind.Everyproperty
ofthisobjectwhichcanberepresentedasatable(ListorDictionary)inheritsallpossibilities
ofthemastertable.WecanmakeagainfavoriteoperationslikeForward,BackwardSkip,
Remove,Addetc.andthesamewithsubnestedtablesandsubsub....subnestedtables.
Toinsertatableinavalueweneed64bytesitsasizeoftableroot.
Table"t1"
Key|Value
1|/...64byte..../
/...64byte..../
/...64byte..../
Key<int>Value
Key<string> Value
1
/...64byte..../
a5
/...64byte....//...64byte..../
2
/...64byte....//...64byte..../ b6
string
3
t7
int
h8
long
2|/...64byte..../
3|/...64byte....//...64byte..../extradata/...64byte..../extradata/...64byte..../
Note,
itsnotpossibletocopythetablewhichhasinvaluesnestedtableswiththe
techniquesdescribedbefore(simplebytescopying).Butitispossibletoautomatethis
process,becausethetableroothasamarkdbreeze.tiesky.comalwaysstartingatthe
samepointfromtablerootstart,alsotherootlengthisfixedwith64bytes,soonedaywe
willmakethisrecursivecopyfunction.
Note
,wearestillthinkingaboutthemethodsnameswhichweusewhilefetchingnested
tablesandweknowthatthetimewillplacecorrectemphasisherealso.
Fractaltablesstructure.GettingStarted
Everyoperationstartsfromthemastertable.Mastertableisatablewhichisstoredinthe
Schemeandyouperfectlyknowitsname.
tran.Insert<int,string>(t1,1,Hello)
tran.Insert<int,string>(t1/Points,1,HelloAgain)
t1andt1/Pointsaremastertables.
So,letsassumewehavemastertablewiththenamet1.Keysofthistableareofinteger
type.Valuescanbedifferent.
tran.Insert<int,string>("t1",1,"hello")
tran.Insert<int,byte[]>("t1",2,newbyte[]{1,2,3})
tran.Insert<int,decimal>("t1",3,324.34M)
Ifyouknowwhatisstoredunderdifferentkeysyoucanalwayscorrectlyfetchthevalues,on
thelowestleveltheyarealwaysbyte[]bytearray.
Toinsertatablewehavedesignednewmethod
tran.InsertTable<int>("t1",4,0)
youneedtosupplyonetypeforkeyresolving,valuewillbeautomaticallyresolvedasbyte
array.Asparametersyouneedtosupplymastertablename,key(4inourexample)and
tableindex.
Asyourememberwecanputmorethen1tableinthevalueandeveryofitwillreside64
bytes.
So,ifindex=0thentablewillresidevaluebytesfrom063,ifindex=1thentablewillreside
valuebytesfrom64127etc....
Inbetweenyoucanputyourownvalues,justremembernottooverlapnestedtablesroots.
Again,wecansay
tran.InsertPart<int,int>("t1",4,587,64)
Key4willhave64bytesofatableandthen4reservedbytesforthevalue587.Youcan
workseparatelywiththem.
Note
,method
InsertTable
givesusextraloadtellingthatwewanttoinsert/change/modify.If
the
table
didntexistinthatplaceitwillbe
automaticallycreated
.AlsoInsertTablewill
notifythesystemthatthread,whoisusingit,triestomodifytablet1,thatswhyall
necessarytechniquesliketran.SynchronizeTables,ifyoumodifymorethenonemaster
table,mustbeused.Theyaredescribedinpreviouschapters.
Wehaveanothermethod
tran.SelectTable<int>("t1",4,0)
InoppositetoInsertTableiftableisnotfounditwillnotbecreated.
Note
,method
SelectTable
willnotcreatetableifitdoesntexistandthismethodis
recommendedfor
READINGTHREADS.
ButalsocanbeusedbyWRITINGthreadsjustto
getthetablewithoutitscreation.
Note,
tran.
InsertTable
and
SelectTable
alwaysreturnvalueoftype
DBreeze.DataTypes.NestedTable
NestedTable
repeatsbyfunctionalityTransactionclassinthescopeoftableoperations.You
willfindthereallwellknownmethods:SelectSelectForwardBackward,Insert,InsertPart,
RemoveKey,RemoveAlletc.
Firstdifferenceisthatyoudontneedtosupplytablenameasparameter.
KeyValue
1
2
3
4/*....64byte...table*//*4bytesinteger*/
KeyValue
1Hi1
2Hi2
3Hi3
Tobuildupsuchstructurewedofollowingcode:
tran
.InsertTable<int>("t1",4,0)
.Insert<int,string>(1,"Hi1")
tran.Commit()
ThisfunctionalprogrammingtechniqueispossibleduetoreturnsofInsertItreturnsthe
underlyingNestedTable.
Toreadthedatawedofollowing:
tran
.SelectTable<int>("t1",4,0)
.Select<int,string>(1)
.PrintOut()
WewillreceiveHi1
PrintOutisasmallconsoleouthelperforcheckingthecontent.
Letsiterate
foreach(varrowintran
.SelectTable<int>("t1",4,0)
.SelectForward<int,string>()
row.PrintOut()
Note,ifyoutrytoInsertintonestedtableaftermasterSelectTableyouwillreceivean
exception.Inserting(Removing,changingetcallmodifications)intoallnestedtables
generationsisallowedonlystartingfrommasterInsertTablemethod.
Letstrymorecomplexstructure
KeyValue
1
2
3
4/*....64byte...table*/
KeyValue
1Hi1
2/*....64byte...table*//*....64byte...table*/
KeyValue
KeyValue
1Xi17Piar7
2Xi28Piar8
3Hi3
varhorizontal=
tran
horizontal.Insert<int,string>(1,"Hi1")
horizontal
.GetTable<int>(2,0) //weuseittoaccessnexttablegenearation
.Insert(1,"Xi1")
.Insert(2,"Xi2")
horizontal
.GetTable<int>(2,1)
.Insert(7,"Piar7")
.Insert(8,"Piar8")
horizontal.Insert<int,string>(3,"Hi1")
//Hereallvaluesforallnestedtableswillbecommitted
tran.Commit()
//Fetchingvalue
tran.SelectTable<int>("t1",4,0)
.GetTable<int>(2,1)
.Select<int,string>(7)
.PrintOut()
//ReturnwillbePiar7
Note
,thereisnoseparateCommitorRollbackofthenestedtablestheyaredoneviamaster
tableCommitorRollback.
[20120525]
SelectreturnsDBreeze.DataTypes.Row
This
Row
weknowfrompreviousexamples,butnowitsenhancedwithnewmethod
GetTable(uinttableIndex),whereyoucangetnestedtablestoredinsideofthisrowby
tableIndex.Itworksformasterandfornestedtables.
{
.Insert<uint,string>(1,"Test1")
tran.Commit()
//foreach(varrowintran.SelectTable("t1",1,1))alsopossiblebut...
foreach(varrowintran.SelectForward<int,byte[]>("t1"))
{
foreach(varr1inrow.GetTable(1).SelectForward<uint,string>())
r1.PrintOut()
}
}
}
//Resultwillbe
1Test1
2Test2
3Test3
InsertDictionary.SelectDictionary.InsertHashSet.SelectHashSet
Wehavecreatedextrainsertandselectstatementsformastertableandnestedtableto
supportdirectcastsoftheDBreezetablesasaC#DictionaryandHashSet(listofunique
keys).
Dictionary<uint,string>_d=newDictionary<uint,string>()
_d.Add(10,"Hello,myfriends")
_d.Add(11,"Sehrgut!")
Dictionary<uint,string>_b=null
//InsertintoMasterTableRow
tran.InsertDictionary<int,uint,string>("t1",10,_d,0,true)
//InsertintoNestedTableDictionary
.InsertDictionary<int,uint,string>(10,_d,0,true)
tran.Commit()
//Selectfrommastertable
_b=tran.SelectDictionary<int,uint,string>("t1",10,0)
_b=tran.SelectTable<int>("t1",15,0)
.SelectDictionary<int,uint,string>(10,0)
tran.InsertDictionary<int,uint,string>("t1",10,_d,0,true)
willcreatefollowingstructure:
t1
Key<int>Value<byte[]>
1
2
..
10/*063bytesnewtable*/
Key<uint>Value<string>
10Hello,myfriends
11Sehrgut!
tran
.InsertDictionary<int,uint,string>(10,_d,0,true)
willcreatefollowingstructure:
t1
1
2
..
......
Key<uint>Value<string>
10Hello,myfriends
11Sehrgut!
Selectwillbeusedtogetthesevalues,Hashsethasthesamesemantic.
Note
,thereisoneimportantflaginInsertDictionaryandInsertHashSet.Itslastparameter
boolwithValuesRemove.
IfyousuppliedbeforeDictionarywithkeys1,2,3....commit......thennexttimeyousupply
Dictionarywithvalues2,3,4
ifwithValuesRemove=true
thenindbwillstaykeys2,3,4
ifwithValuesRemove=false
thenindbwillstaykeys1,2,3,4
Thesestructuresdesignedashelpfunctionsfor:
Thequickmethodtostoreasetofkeys/valuesintothenestedtablesfromDictionaryor
HashSet(InsertDictionary(....,false)).
HelpfunctionsforsmallDictionaries/HashSetstobestoredandSelectedwithautomatic
removalandupdate(InsertDictionary(....,true)).
AbillitytogetthefulltableofanyKey/ValuetypeasDictionaryorHashSetrightin
memory.
[20120526]
WehavealsoaddedInsert/SelectDictionary/HashSetforthetablesthemselves(notjust
movedbylevels)
Wecanmakefollowing:
insertingrightintot1tablevaluesrepresentedasDictionary:
tran.InsertDictionary<int,int>("t1",newDictionary<int,int>(),false)
insertingintot1row1atablewhichlocatesfrom0byteofrowaDictionary:
tran.InsertTable<int>("t1",1,0).InsertDictionary<uint,uint>(new
Dictionary<uint,uint>(),false)
Correspondingselects:
tran.SelectDictionary<int,int>("t1")
tran.SelectTable<int>("t1",1,0).SelectDictionary<uint,uint>()
ThesameforHashSets.
[20120529]
Nestedtablesmemorymanagement.
Wehaveasituationofmemorygrowthincaseifweuselotsofnestedtablesinside
ofonetransaction.Supportofatabletakesamemoryamount.
Mastertableandnestedintoittablessharethesamephysicalfile.Currentengine
automaticallydisposesmastertableandallnestedtableswhentransaction(workingwith
mastertable)isfinished.Butonlyincasewhenparallelthreadsdontreadfromthesame
tableinthesametime.Mastertableandnestedintoittableswillbedisposedtogetherwith
thelastworkingwiththistabletransaction.Ifwewriteintothetableonceper7secondsand
readonceper2seconds,definitelythistablewillbeabletofreeresidingmemory
inbetween.
Somemoresituations.Forexampleweinsertdatainsuchmanner:
{
for(inti=0i<100000i++)
{
tran.InsertTable<int>("t1",i,1)
.Insert<uint,uint>(1,1)
}
tran.Commit()
}
Reallybadcaseforthememory.Inthiscasewehavetoopen100000+1(master)tablesand
holdtheminmemorytilltran.Commit()
Inourtestsusedmemoryhasgrownupfrom30MB(basicrunofatestprogram)upto
350MB...aftertransactionwasfinishedtheprocesssizedidntchange,butthose320MB
weremarkedtobecollectedby.NETGarbageCollector,socallingGC.Collect(orusingthe
processfurther)bringsbackto30MB.
Andfornowitshardtofindoutthewayshowtoavoidthismemorygrowth.Itsnotsocritical
whenyouinsertinsmallchunks(100records).Soyoumustrememberaboutthat.
Anothercase:
Looksevenmoreinteresting.Whenweselectdata
{
for(inti=0i<100000i++)
{
varrow=tran.SelectTable<int>("t1",i,1)
.Select<uint,uint>(1,1)
if(row.Exists)
{
//..do
}
}
}
Here,aftereveryloopiterationwedontneedanymoreusedtable,butitstillstaysin
memoryandmakeitgrowing.Inthisexamplememoryhasgrownupfrom30MBupto
135MB,sureifyouselectmorerecordsitwillneedmorememoryresource.
Exactlyforsuchcasewehadtointegratetable.Closemethod.
TouseClose,weneedavariableforaccessingthistable.Ourcodewilllooklikethisnow:
{
foreach(varrowintran.SelectForward<int,byte[]>("t1"))
{
vartbl=row.GetTable(1)
if(!tbl.Select<uint,uint>(1).Exists)
Console.WriteLine("not")
tbl.CloseTable()
}
}
Nowmemoryholdsthenecessarylevel.
Note
,WhenwecallNestedTable.Close,method,wewanttoclosecurrenttableandall
nestedinittables.EverymastertableInsertTableorSelectTable(and
nestedTable.GetTable)increaseopenquantityvariableby1,everyCloseTabledecreses
valueby1,whenvalueislessthen1,thenthetablewithallnestedinittableswillbe
closed.
Ifweforgettoclosetablethenitwillbeopentillalloperationswithmastertablearefinished
andautomaticdisposeworks.
Note,NestedTable.DisposecallsCloseTableautomatically,
sowecanmake:
{
using(vartbl=row.GetTable(1))
{
if(!tbl.Select<uint,uint>(1).Exists)
Console.WriteLine("not")
}
}
}
Rules.
DontclosethetablebeforeyouCommitorRollbackit.
Transactionendwillclosemasterandnestedtablesautomaticallyifnoother
threadsareworkingwithit,probablyparallelthreadwillcloseitafterfinish.
Closetableinstancesmanuallyifoperationswiththetableareveryintensiveand
thereisnochancethatitwillbeclosedautomatically.
ControlInsertTablethesamewayasSelectTable.
Itspossibletoclosetablesofallnestinggenerations,dependinguponyourtable
structure.Theywillbeclosedstartingfromcalledgeneration.
Thischapterisontheleveloftheexperiment.
SecondaryIndex.Directkeyselect.
Herewepresentanotherexperimentalapproach.
Ifweneedtosupportotherindicesthenourtablekey,wherewestoreourobjectsweneed
tocreateothertableswherekeyswillbesecondaryindexetc.Insecondaryindextablewe
canstoredirectpointerthefirsttablewiththeobjectincontrastwiththekey.
Whenweinsertorchangethekeywehaveanabilitytoobtainitsfilepointer:
byte[]ptr=null
{
tran.Insert<int,int>("t1",12,17,outptr)
tran.SelectDirect<int,int>("t1",ptr).PrintOut()
tran.ChangeKey<int>("t1",12,15,outptr)
tran.SelectDirect<int,int>("t1",ptr).PrintOut()
}
thenwecangetthevaluebypointereconomizingtimeforthesearchofthefirsttablekey.
Note
,whenweupdateprimarytable,whoholdsfullinformationabouttheobject,itspointer
canbemoved,thatswhyourDALmustupdatevalue(pointertotheprimarytablekey)in
thesecondarytablealso.Whenwedeletefromprimarytable,wemustdeleteinthesame
transactionfromsecondaryindextablealso.
Thesamewecanmakeinsideofnestedtables.
Note,fornestedtablesSelectDirectmustbeusedexactlyfromthetablewhereyouare
searchinginformationtoavoidcollisions:
byte[]ptr=null
using(vartbl=tran.InsertTable<int>("t3",15,0))
{
tbl.Insert<int,int>(12,17,outptr)
tran.Commit()
using(vartbl=tran.SelectTable<int>("t3",15,0))
{
varrow=tbl.SelectDirect<int,int>(ptr)
row.PrintOut()
Note
,wecangetpointertothevalueinsideof
Insert
,
InsertPart
and
ChangeKey
for
primaryandnestedtables.
[20120601]
DynamiclengthdatablocksandbindingthemtoRow.Column.
Insideofthetablewehavekeyandvalue.Iftothinkaboutthevalueasrowwithcolumns,
thatgivesusabilitytostoreinonerowindependentdatatypes,whichwecanaccessusing
Row.GetValuePart(uintstartIndex,uintlength)andeverythingseemstobegood,whenour
datatypeshavefixedlength.Butsometimesweneedtostoreinsideofcolumns
dynamiclengthdatastructures.
Forthiswehavedevelopedfollowingmethodinsideofthetransactionclass:
publicbyte[]InsertDataBlock(stringtableName,byte[]initialPointer,byte[]data)
Datablocksliveinparallelwiththetableitselfandinheritthesamedatavisibilitybehaviour
fordifferentthreadslikeotherstructures.
NestedtablesalsohaveInsertDataBlockmethod.
Note
,InsertDataBlockalwaysreturnbyte[]ofthesamelength16bytesitsadefinitionof
thestoredvalue,becausereturnedvaluelengthisfixedwecanuseitascolumninsideofa
Row.
Note,
if2parameterinitialPointerisNULLthennewdatablockwillbecreatedforthetable,if
notNULLitcanmeanthatsuchdatablockalreadyexistsandDBreezewilltrytooverwriteit.
Note,datablocksobeytransactionrules,sotillyoucommitupdateddatablock,parallel
readingthreadswillcontinuetoseeitslastcommittedvalue.Wecanalsorollbackchanges.
Afterweinsertdatablockwewanttostoreitspointerinsideofarow,tohaveanabilitytoget
itlater:
byte[]dataBlockPtr=tran.InsertDataBlock("t1",null,newbyte[]{1,2,3})

herewehavereceiveddatablockpointerandwewanttostorethispointerint1row
tran.InsertPart<int,byte[]>("t1",17,dataBlock,10)
wehavestoredpointertothedatablockinsideoft1key(17)startingfromindex10,pointer
hasalwaysfixedlength16byte,startingfromindex26wecangoontostoreothervalues.
Nowwewanttoretrievethedataback:
ItspossibleviaRowobject:
varrow=tran.Select<int,byte[]>("t1",17)
byte[]res=row.
GetDataBlock
(10)
Note,DataBlockcanstorenullvalue.
Updated:
Also,wecannowdirectlygetDataBlocksfromtransaction:
//Whendatablockissavedinmastertable
tran.SelectDataBlock("t1",dataBlockPointer)
//Whendatablockissavedinnestedtable
tran.SelectTable<int>("t1",1,0).SelectDataBlock(dataBlockPointer)
Ifwewanttostorelinktothedatablockinsideofnestedtablerow,wemustmakeitvia
NestedTablemethod:
vartbl=tran.InsertTable<int>("t1",18,0)
byte[]dbp=tbl.InsertDataBlock(null,newbyte[]{1,2,3})
tbl.InsertPart<int,byte[]>(19,dbp,10)
tran.Commit()
tbl.CloseTable()
tbl=tran.SelectTable<int>("t1",18,0)
varrow=tbl.Select<int,byte[]>(19)
byte[]fr=row.GetDataBlock(10)
if(fr==null)
Console.WriteLine("T1NULL")
else
Console.WriteLine("T1"+fr.ToBytesString())
Systemunderstandsemptypointerstothedatablock.Infollowingexamplewetrytoget
notexistingdatablock,thenupdateitandwritepointerback:
varrow=tran.Select<int,byte[]>("t1",17)
byte[]dataBlock=row.GetDataBlock(10)
dataBlock=tran.InsertDataBlock("t1",dataBlock,newbyte[]{1,2,3,7,8})
tran.InsertPart<int,byte[]>("t1",17,dataBlock,10)
tran.Commit()
HashFunctionsofcommonusage.Fastaccesstolongstringsandbytearrays.
DBreezesearchtrieisavariationofradixtrie,optimizedbyallparameters
LianaTrie
.
So,ifwehavekeysoftypeint(4bytes),wewillneedfrom1upto4HDDhitstogetrandom
key(wedonttalkaboutHDDpossibleproblemsandOSfilesystemfragmentationshere).If
wehavekeysoftypelong(8bytes)wewillneedfrom1upto8hits,dependinguponkeys
quantityandcharacter.Ifwestorelongerbytearrays,wewillneedfrom1uptomaxlength
ofthebiggestkeyhits.Ifwestoreinonetable4suchstringkeys:
key1:
http://google.com/hi
key2:
http://google.com/bye
key3:
http://dbreeze.tiesky.com
key4:abrakadabra
togetrandomlykey1wewillneed
http://google.com/
h19hits
http://google.com/
b19hits
http://
d8hits
togetrandomlykey4wewillneedonly1hit
(afteryoufindakeyinrangeselects,searchingofothers,insideofiterationwillworkfast)
So,ifweneedtouseStartsWith,orweneedsortingofsuchtable,wehavetostorekeyslike
theyare.
Butifweneedjustrandomaccesstosuchkeys,thebestapproachwillbetostorenotthe
fullkeysbutonlytheir4/8or16bytesHASHCODES.Also,hashedkeysandvalueswith
directphysicalpointers,canrepresentsecondaryindex.Forexample,infirsttablewestore
keys,liketheyare,withthecontentandinsecondtablewestorehashesofthosekeysand
physicalpointerstothefirsttable.Nowwecangetsortedviewandhavefastestrandom
access(from1upto8hits,ifhashisof8bytes).
Hashescanhavecollisions.WehaveintegratedintoDBreezesourcesMurMurHash3
algorithm(whichreturnsback4byteshash)andaddedtwomorefunctionstoget8bytes
and16byteshashcode.Werecommendtousethose8bytesor16bytesfunctionstostay
collisionsafewithaveryhighprobability.Ifyouneed1000%guarantee,usenestedtable
undereveryhashandstoreinitrealkey(orkeysincaseofcollisions),forcheckingorsome
kindofothertechnique,likeserializedlistofkeyswiththesamehashcode.
DBreeze.Utils.Hash.MurMurHash.MixedMurMurHash3_648bytereturnsulong
and
DBreeze.Utils.Hash.MurMurHash.MixedMurMurHash3_12816bytereturnbyte[]
[20120628]
RowhaspropertyLinkToValue(actuallyitsalinktoKey/Value),forgettingdirectlinktothe
rowandusingittogetherwithSelectDirect.Alllinks(pointerstokey/valuepairs)nowreturn
fixed8bytesandcanbestoredasvirtualcolumninrows.
Also,wecannowdirectlygetDataBlocksfromtransaction:
//Whendatablockissavedinmastertable
tran.SelectDataBlock("t1",dataBlockPointer)
//Whendatablockissavedinnestedtable
tran.SelectTable<int>("t1",1,0).SelectDataBlock(dataBlockPointer)
[20120905]
Integratedincrementaldatabasebackupability.
Tomakeitworkinginstantiatedbreezelikethis:
DBreezeConfigurationconf=newDBreezeConfiguration()
DBreezeDataFolderName=@"D:\temp\DBreezeTest\DBR1",
Backup=newBackup()

BackupFolderName=@"D:\temp\DBreezeTest\DBR1\Bup",
IncrementalBackupFileIntervalMin=30
engine=newDBreezeEngine(conf)
IfBackupobjectisnotincludedinconfigurationorDBreezeisinstantiatedwithout
configuration,likeitwasbefore,incrementalbackupwillbeswitchedoff.Sure,thereisstill
DBreezeconstructorwithoutconfigurationparameteratall.
Ifyouhaveexistingdatabasesyoucanmakeitsfullcopy(snapshot)andstarttocontinue
toworkwiththeincrementalbackupoptionswitchedon.Backupwillcreateonceper
IncrementalBackupFileIntervalMinanewfile(oldfilesarereleasedandcanbecopiedout
anddeleted).Currentbackupfileisalwayslockedbydbreeze.Youhavetospecifyfolderfor
dbreezeincrementalbackupfilesBackupFolderName.Thatsall.
Ifyoustartnewdatabasewithincrementalbackupoption,thenlateryouwillbeableto
recreatethewholedbfrombackupfiles,ifyouhavestartedfromasnapshotthenbackup
filescanbringyoursnapshottocurrentdbstate.
Youcanrestorebackupinthefolderwhereyoursnapshotresidesor,ifincrementalbackup
wasswitchedonfromthebeginning,intotheemptyfolder.
ExampleofbackuprestorationisshowninVisualTestersatelliteprojecttoDBreeze
solution,underbuttonRestoreBackup.
SwitchedonincrementalbackupoptionbringstoWritespeeddecrease,Readspeedis
untouched.
Insertingonemillionofintegerswithoutbackupoption9secwithoption17sec.
[20120922]
!!!!!!!!!!!!!!!!!!!!!!!!IMPORTANTWHOUSEDDATABASETILLTHISPERIOD
!!!!!!!!!!!!!!!!!!!!!!!!ANDUSEDDBreeze.UtilsByteProcessingextensions:DateTime
!!!!!!!!!!!!!!!!!!!!!!!!.To_8_bytes_array_BigEndian()and
!!!!!!!!!!!!!!!!!!!!!!!!byte[]To_DateTime_BigEndian()
AfterattachingnewDBreezeandrecompilationoftheprojectyouwillseeerrors,because
suchfunctionsdontexistanymoreinDBreeze.
Why?
Itsanissue,historicalissue.OurDBreezegenerictypeconverter(weuseitin
tran.Insert<DateTime,DateTime..tran.InsertPart<DateTimeetc.)waswrittenbeforesome
ByteProcessingUtilsfunctionsandsomehowDateTimewasconvertedfirstto
ulong
and
thentobyte[].Otherwise,To_DateTime_BigEndian()andTo_8_bytes_array_BigEndian()
fromDBreeze.Utilsused
long
,suchunpleasantthing.
Well,nowwhat?
So,wehavedecidedtoleaveDateTimeconvertertoworkwithulong.Itdoesnthave
influenceonthespeed,andwedontneedtorecreatemanyexistingdatabases.
WehavecreatedinsteadsuchfunctionsinDBreeze.Utils.ByteProcessing:
publicstatic
DateTimeTo_DateTime(thisbyte[]value)
andthiswillworkwith
ulong
and
publicstatic
byte[]To_8_bytes_array(thisDateTimevalue)
whichrecreatesDateTimefrom8byte
array.
Withthisfunctionswerecommendtoworkinthefuture
.Thesamealgorithmsare
usedbygenericconverter.
But,ifyouhavealreadyusedmanualDateTimeconversions,wehavelefttwofunctionsfor
compatibility:
publicstaticbyte[]
To_8_bytes_array_zCompatibility
(thisDateTimevalue)
(thisyoumustputinthecodeinsteadofoldTo_8_bytes_array_BigEndianconcerning
DateTime)and
DateTime
To_DateTime_zCompatibility
(thisbyte[]value)(thisyoucanuseinsteadofold
To_DateTime_BigEndian)
TheybothgoontoworkwithDateTimeaslongtobyte[].
So,thinkaboutthatanddowhatyoushoulddo:)
Actually,nothingshouldstopusonthelightwayoftheGodsLove!
Storinginthevaluecolumnsofthefixedsize.
Forthelastsomemonthswehavecreatedmanytableswithdifferentvalueconfigurations,
combiningwaysofthedatastorage.Oneofthemostpopularwayishandlingvaluebyte[]as
setofcolumnsoffixedlength.Wefoundoutthatwehavelackofnullabledatatypesandfor
thiswehaveaddedinDBreeze.Utils.ByteProcessingarangeofextensionsforallstandard
nullabledatatypes:
Youtakeanystandardnullabledatatypeint?,bool?,DateTime?,decimal?,float?uint?etc.
andconvertitintobyte[]usingDBreeze.Utilsextensions:
publicstaticbyte[]To_5_bytes_array_BigEndian(thisint?value)
or
publicstaticbyte[]To_16_bytes_array_BigEndian(thisdecimal?input)
etc...
andthesamebackward:
publicstaticDateTime?To_DateTime_NULL(thisbyte[]value)
or
publicstaticushort?To_UInt16_BigEndian_NULL(thisbyte[]value)
etc.withNULLintheend
Note,thatpracticallyallnullableconverterscreatebyte[]on1bytelongerthennotnullable.
SometimesinonevalueweholdsomecolumnsoffixedlengththensomeDataBlocks,which
representpicturesorsoandthenDataBlockswhichrepresentbigtextorjsonserialized
objectparts.Butwefoundout,thatwemissstoringoftextintheway,likestandardRDBMS
makethat:nvarchar(50)NULLorvarchar(75).SurewecanuseDataBlocksforthat,but
sometimeswedontwantit,especiallyhavingthatDataBlockreferencewillreside16bytes.
WehaveaddedinDBreeze.UtilsByteProcessingtwomoreextensions:
publicstaticbyte[]To_FixedSizeColumn(thisstringvalue,shortfixedSize,bool
isASCII)
and
publicstaticstringFrom_FixedSizeColumn(thisbyte[]value,boolisASCII)
TheybothwillemulatebehaviourofRDBMStextfieldsofthefixedreservationlength.
Maximum32KB.Minimum1byteforASCIItextand4bytesforUTF8text.
Takeastring(itcanbealsoNULL)andsay:
stringa=mytext
byte[]bta=a.
To_FixedSizeColumn(50,true)
andyouwillreceivebytearrayof50+2=
52bytes
thisyoucanstoreinyourvaluefrom
specificplace(letssay10).
Note,returnedsizewillbealways2byteslongerweneedthemtostorelengthofthereal
textinsideofthefixedsizearrayandNULLflag.
Thentakeyourvalue.Substring(10,52).
From_FixedSizeColumn(true)
andyouwillreceive
yourmytext.isASCIImustbesettofalseifyoustoreUTF8value.Ifsizeofthetext
exceedsthefixedSizeparameter,thenvaluewillbetruncated(correctalgorithmisused,so
onlyfullUTF8charswillbestoredwithoutanygarbagebytesintheend).
Sometimes,itsveryusefulasafirstbyteofthevaluetosetuparowversion,then,
dependinguponthisversion,thefurthercontentofthevaluecanhavedifferent
configurationsofthecontent.
[20121012]
Behaviouroftheiteratorswiththemodificationinstructionsinside.
Letsassumethatbeforeeveryfollowingexample,wedeletetablet1andthenexecute
suchinsert:
{
for(inti=200000i<800000i++)
{
tran.Insert<int,int>("t1",i,i)
}
tran.Commit()
}
Sometimesitsinterestingforustomaketablemodificationswhileiteration,likehere:
{
//t1isnotinmodificationlist,enumeratorsvisibilityscopeisparallelread
foreach(varrowintran.SelectForward<int,int>("t1"))
{
tran.RemoveKey<int>("t1",row.Key)
tran.Commit()
insuchexampleitwillworkgood.
Inthenextexampleitwillalsowork:
{
tran.SynchronizeTables("t1")
//t1isinmodificationlist,enumeratorsvisibilityscopeissynchronizedread/write
//probablywecanseechangesmadeinsideofiterationprocedure.
var
en
=tran.SelectForward<int,int>("t1").
GetEnumerator
()
while(
en
.
MoveNext
())
{
tran.RemoveKey<int>("t1",en.Current.Key)
}
tran.Commit()
}
Enumeratoren,referstowritingrootatthismoment,becauseourtablewasaddedinto
modificationlist(bySynchronizeTableoranyothermodificationcommand,likeinsert,
removeetc...),andchangesofthetable,evenbeforecommitting,canbereflectedinsidethe
enumerator.
But,wedeletethesamekeywhichweread,thatswhythistaskwillbeaccomplishedgood.
Wedontinsertordeleteelementsofthefutureiterations.
Inthenextexamplewecanhavenotdesiredbehaviour:
{
intpq=799999

varen=tran.SelectForward<int,int>("t1").GetEnumerator()
while(en.MoveNext())
{
tran.RemoveKey<int>("t1",
pq
)
pq
}
tran.Commit()
}
Wewillnotdeleteallkeysinthepreviousexample.Enumeratorwillstoptoiterate
somewhereinthemiddle,whereexactlydependsuponkeystructureandnotreallyuseful
forus.
So,ifyouaregoingtoiteratesomethingandchangepossibleelementsofthefuture
iterations,thereisnoguaranteeforthecorrectlogicexecution.Thisconcernssynchronized
iterators.
Tomakeitcorrect,wehaveaddedforeveryrangeselectfunctionanoverloadwiththe
parameter
boolAsReadVisibilityScope
.Itconcernsnestedtablesrangeselectfunctions
also.
Nowwecanmakesomethinglikethis:
{
intpq=799999
varen=tran.SelectForward<int,int>("t1"
,true
).GetEnumerator()
while(en.MoveNext())
{
tran.RemoveKey<int>("t1",
pq
)
pq
}
tran.Commit()
}

Allkeyswillbedeletedcorrectly.Becauseourenumeratorsvisibilityscopewillbethesame
asinparallelthread,soitwillseeonlycommitteddataprojection,beforethestartofthe
currenttransaction.
Nowwecanvarywhichvisibilityscopefortheenumerator,whosetableisinsideof
modificationlist,wewanttochoose,synchronizedorparallel.Defaultrangeselects,without
extraparameter,iftableisinmodificationlistwillreturnsynchronizedview.
[20121015]
SecondaryIndexes.Goingdeeper.
Transaction/NestedTablemethod
Select
nowisalsooverloadedwith
bool
AsReadVisibilityScope,
forthesamepurposesasdescribedinthepreviouschapter.
Letsassumethatwehaveanobject:
publicclassArticle
{
[PrimaryKey]
publiclongId=12
publicstringName=A1
[SecondaryKey]
publicfloatPrice=15f
}
PrimaryandSecondarykeysattributes,fornow,dontexistinDBreeze.Butideaisfollowing:
fromfieldIdwewanttomakePrimaryindex/keyandfromfieldPricewewanttocreate
oneofoursecondaryindexes.
FornowDBreezedoesnthaveextraobjectlayer,sowewouldmakesuchsaveinthe
followingformat:
usingDBreeze
usingDBreeze.Utils
publicvoidSaveObject(Articlea)
{
byte[]ptr=null
{
//InsertingintoPrimaryTable
tran.Insert<long,byte[]>
(Article,
a.Id.To_8_bytes_array_BigEndian(),
//Idprimarykey
a.Name.To_FixedSizeColumn(50,false)
//letitbenotDataBlock
.Concat(
a.Price.To_4_bytes_array_BigEndian()
),
outptr
//gettingbackaphysicalpointer
)
//InsertingintoSecondaryIndextable
tran.Insert<byte[],byte[]>
(ArticleIndexPrice,
a.Price.To_4_bytes_array_BigEndian()
//compoundkey:price+Id
.Concat(
a.Id.To_8_bytes_array_BigEndian()
,
ptr
//valueisapointertotheprimarytable
)
)
tran.Commit()
}
}
Somethinglikethis.Inthereallifeallprimaryandsecondaryindexescouldbepackedinto
thenestedtablesofoneMasterTableunderdifferentkeys.
Wehavefilled2tables.FirstisArticle.AskeytherewestoreArticle.Idasvaluewestore
articlenameandprice.SecondtableisArticleIndexPrice.Itskeyisconstructedfrom
(float)Price+(long)ArticleIditsunique,sortable,comparableandsearchable.Such
techniquewasdescribedinpreviousarticles.Asavaluewestorephysicalpointertothe
primarykeyinsideoftheArticletable.Whenwehavesuchphysicalpointer,searchingof
Key/ValueofthePrimaryTableArticleisonlyoneHDDhit.
Butkeysandvaluesarenotalwaysstatic.Sometimesweremovearticles,sometimeswe
changethepriceorevenexpandthevalue(inthelastcase,weneedtosavenewphysical
pointerintosecondaryindextable).
IfweremoveArticle,wemustremovecompoundkeyfromthetableArticleIndexPricealso.
Whenweupdateprice,insideoftableArticle,wemustdeleteoldcompoundkeyfromthe
tableArticleIndexPriceandcreatenewone.

Itmeans,thateverytimewhenweinsertsomethingintotableArticleitcanbecountedasa
probableupdate,andwemustcheck,ifrowwithsuchIdexistsbeforeinsert.Ifyesthenwe
mustreadit,deletecompoundkey,constructandinsertnewcompoundkeyintothetable
ArticleIndexPriceandfinallyupdatevalueinthetableArticle.
Thisallcanslowdowninsertprocessverymuch.
Thatswhywehaveaddedforeverymodificationcommand,insideofthetransactionclass
andnestedtableclass,usefuloverloads:
Modificationcommandsoverloads(thesamefornestedtables):
publicvoid
Insert
<TKey,TValue>(stringtableName,TKeykey,TValuevalue,outbyte[]
refToInsertedValue,
outboolWasUpdated
)
publicvoid
InsertPart
<TKey,TValue>(stringtableName,TKeykey,TValuevalue,uint
startIndex,outbyte[]refToInsertedValue,
outboolWasUpdated
)
publicvoid
ChangeKey
<TKey>(stringtableName,TKeyoldKey,TKeynewKey,outbyte[]
ptrToNewKey,
outboolWasChanged
)
publicvoid
RemoveKey
<TKey>(stringtableName,TKeykey,
outboolWasRemoved
)
Actually,Dbreeze,wheninsertsdata,knows,ifitsgoingtobeanupdateornewinsert.
ThatswhyDbreezecannotifyusaboutthis.
Wegoontoinsertdatainusualmanner.Ifflag
WasUpdated
equalstotrue,thenweknow
thatitwasanupdate.Wecanuseournew,
overloaded
withvisibilityscopeparameter,
Select
togetkey/valuepair,whichwasbeforemodificationandchangesecondaryindex
table.Weneedtomakethisactiononlyincaseofupdate/remove/changecommand,butnot
incaseofthenewinsert.
[20121016]
SecondaryIndexes.Goingdeeper.Part2
IfwestoreinsideofvalueDataBlocks(notjustserializedvalueorcolumnsoffixedlength),
beforewemakeanupdateofsuchvalue,wemustreaditinanycasepreviousvaluecontent
(togetDataBlocksinitialpointersforupdates).So,againeveryinsertcanbecountedas
probableupdate.Followingtechnique/benchmarkshowsustimeconsumptionforreading
previousrowvalueversionbeforeinsert:
Thisisastandardinsert:
byte[]ptr=null
boolwasUpdated=false
DBreeze.Diagnostic.SpeedStatistic.StartCounter("a")
for(inti=200000i<800000i++)
tran.Insert<int,int>("t1",i,i,outptr,outwasUpdated)
}
DBreeze.Diagnostic.SpeedStatistic.PrintOut("a",true)
tran.Commit()
Operationtook
9300ms(9sec2012y,1.5sec2015y)
.
1MLN
of
inserts
.
Thisisaninsertwithgettingpreviousrowversionbeforeinsert:
{
byte[]ptr=null
DBreeze.DataTypes.Row<int,int>row=null
DBreeze.Diagnostic.SpeedStatistic.StartCounter("a")
for(inti=200000i<800000i++)
{
//Note,weuseSelectwithVisibilityScope=ParallelRead
row=tran.Select<int,int>("t1",i,
true
)
if(row.Exists)
//doupdate
tran.Insert<int,int>("t1",i,i,outptr)
else
{
//doinsert
tran.Insert<int,int>("t1",i,i,outptr)
}
}
DBreeze.Diagnostic.SpeedStatistic.PrintOut("a",true)
tran.Commit()
}
Thisoperationtook
10600ms(10sec)
.
1MLN
of
inserts,
distinguishingbetweenupdates
andinserts.
Remember,thatDBreezeinsertandselectalgorithmsworkwithmaximumefficiency
inbulkoperations,whenkeysaresuppliedsortedinascendingorder(descendingis
abitslower).So,sortbulkchunksinmemorybeforeinserts/selects.
Previous2exampleswereaboutpureinserts,andwerunthemagainhavingdataalreadyin
thetable,soallrecordshavetobeupdated:
1example1MLNofupdatestook28sec
2example1MLNofupdateswith(gettingrowpreviousversion)took36sec.
[20121023]
Dbreezelikeinmemorydatabase.
Dbreezecanresidealsofullyinmemory.Itsjustafeature.Havingthesamefunctionalityas
diskbasedversion.
Instantiatingexample:
DBreeze.DBreezeEnginememeng=newDBreezeEngine(newDBreezeConfiguration()
Storage=DBreezeConfiguration.eStorage.MEMORY
})
using(vartran=memeng.GetTransaction())
{
for(inti=0i<1000000i++)
tran.Insert<int,int>("t1",i,i)
Console.WriteLine(tran.Count("t1"))
tran.Commit()
Itworksabitslowerthen.NETDictionaryorSortedDictionary,becausehaslotsof
subsystemsinside,whichmustbesupported,anddesignedtoworkwithverylargedata
sets,withoutindexfragmentationaftercontinuousinserts,updatesanddeletes.
Outoftheboxbulkinsertspeedincrease.
WehaveincreasedstandardbulkinsertspeedofDBreeze(about5times),byaddinga
specialmemorycachelayerbeforeflushingdataonthedisk.Bystandardconfiguration,20
tables,whicharewritteninparallel,receivesuchmemorybufferofsize1MBeach,before
diskflush.The21th(andsoon,parallel)willwritewithoutbuffer.Afterdisposingofthe
writingtransactionsothertablescanreceivesuchbuffer,soitsnotboundtothetables
namestablesarechosenautomaticallyrightintimeoftheinsert.
NowDBreeze,instandardconfiguration,canstoreinbulk(ascendingordered)500Krecords
per1seconds(BenchmarkPCistaken).6parallelthreadscouldwriteinto6differenttables
1MLNofrecordseach,forthe3.4seconds,whatwasabout40MB/sand1.7MLNsimple
recordspersecond(seeBenchmarkingdocument).
[20121101]
IterationsSelectBackwardStartsWithClosestToPrefixand
SelectForwardStartsWithClosestToPrefix.
Theybothconcernmasterandnestedtables.
Ifwehaveinthetablestringkeys:
"check
"sam"
"slash
"slam"
"what"
stringprefix="slap"
foreach(varrowintran.SelectForwardStartsWithClosestToPrefix<string,byte>("t1",prefix))
{
Console.WriteLine(row.Key)
}
Result:

slam
slash
andfor
foreach(varrowintran.SelectBackwardStartsWithClosestToPrefix<string,byte>("t1",prefix))
Result:
slash
slam
[20121111]
Alternativetablesstoragelocations.
StartingfromcurrentDBreezeversionweareabletosetuptableslocationsbytablenames
patternsglobally.WecanmixtablesphysicallocationsinsideofoneDBreezeinstance.
Tablescanresideindifferentfolders,ondifferentharddrivesandeveninmemory.
DBreezeConfigurationobjectisenrichedwiththepublicaccessibleDictionary
AlternativeTablesLocations.
NowwecancreateDBreezeconfigurationinthefollowingformat:
DBreezeConfigurationconf=newDBreezeConfiguration()
DBreezeDataFolderName=@"D:\temp\DBreezeTest\DBR1",
Storage=DBreezeConfiguration.eStorage.DISK,
//SETTINGUPALTERNATIVEFOLDERFORTABLEt11
conf.AlternativeTablesLocations.Add(
"t11
",@"D:\temp\DBreezeTest\DBR1\INT")
//SETTINGUPTHATALLTABLESSTARTINGFROMmem_mustresideinmemory
conf.AlternativeTablesLocations.Add("
mem_*
",
String.Empty
)
//SETTINGUPTablepatterntoresideindifferentfolder
conf.AlternativeTablesLocations.Add(
t#/Items
",@"D:\temp\DBreezeTest\DBR1\EXTRA")
engine=newDBreezeEngine(conf)
So,if
value
oftheDictionaryAlternativeTablesLocationskeyis
empty
,tablewillbe
automaticallyforcedtowork
inmemory
.Ifpatternforthetableisnotfound,tablewillbe
created,overridingDBreezemainconfigurationsettings(DBreezeDataFolderNameand
StorageType).
If
onetable
corresponds
tosomepatterns
,the
first
onewillbe
taken
.
PatternslogicisthesameasinTransactionSynchronizeTables:
$*#patternextrasymbols
"U"intersects,!Udoesntintersect
*1ormoreofanysymbolkind(everysymbolafter*willbecutted):Items*U
Items123/Picturesetc...
#symbols(exceptslash)followedbyslashandminimumanothersymbol:Items#/PictureU
Items123/Picture
$1ormoresymbolsexceptslash(everysymbolafter$willbecutted):Items$UItems123
Items$!UItems123/Pictures
Patternscanbecombined:
Items#/Pictures#/Thumbs*
canintersect
Items1/Pictures125/Thumbs44
or
Items458/Pictures4658/Thumbs1000
etc...
Incrementalbackuprestorerworksonthefilelevelandknowsnothingaboutuserslogical
tablenames.Itwillrestorealltablesinonespecifiedfolder.Later,afterstartingDBreezeand
readingthescheme,itspossiblemanuallytoresidedisktablefilesintocorresponded
physicalplacesduetothestoragelogic.
[20130529]
Speedingupbatchmodifications(updates,randominserts)
ToeconomizediskspaceDBreezetriestoutilizethesameHDDspace,ifitspossible,in
caseofdifferenttypesofupdates.
Thereare3placeswhereupdatesarepossible:
Updateofsearchtrienodes(LianaTrienodes)
UpdateofKey/Values
UpdateofDataBlocks
Tobesurethatoverwritingdatafilewillnotbecorruptedincaseofpowerloss,firstwehave
towritedataintorollbackfile,thenintodatafile.DBreezeinstandardmodeexcludesany
OSintermediatecache(onlyinternalDBreezecache)andmakeswritestothebaremetal.
TodaysHDDsandevenSSDsarequiteslowfortherandomwrite.Thatswhyweusea
techniqueofchangingrandomwritesintosequentialwrites.

WhenweuseDBreeze,forstandarddataaccumulationoftherandomdatafromdifferent
sources,insideofsmalltransactions,thespeeddegradeisnotsovisible.Butwecanseeit
verygoodwhenweneedtoupdateabatchofspecificdata.
WeDONTSEESPEEDDEGRADE,when
weinsertbatchofgrowingupkeysanynewly
insertedkeyisalwaysbiggerthanmaximalexistingkey(SelectForwardwillreturnnewly
insertedkeyasthelastone).Forsuchcaseweshoulddonothing.
WeCANSEESPEEDDEGRADE,when
weupdatebatchofvaluesordatablocksorifwe
insertabatchofkeysinrandomorderand,especially,ifthesekeyshavehighentropy.
Forsuchcaseswehaveintegratednewmethodsfortransactionsandfornestedtables:
tran.
Technical_SetTable_OverwriteIsNotAllowed(t1)
or
vartblABC=tran.InsertTable<byte[]>(masterTable,newbyte[]{1},0)
tblABC.
Technical_SetTable_OverwriteIsNotAllowed()
Thistechniqueisinterestingforthetransactionswithspecificbatchmodifications,
wherespeedreallymatters.Onlydevelopercananswerthisquestionandfinda
balance.
Thistechniqueisnotinterestingforthememorybaseddatastores.
Thesemethodsworkonlyinsideofonetransactionandmustbecalledforevery
tableornestedtableseparately,beforetablemodificationcommand.
Whennewtransactionstarts,overwriteautomaticallywillbeallowedagainforall
tablesandnestedtables.
Overwritingconcernsall:searchtrienodes,valuesanddatablocks.
Rememberalwaystosortbatchascendingbykey,beforeinsertitwilleconomize
HDDspace.
Ofcoursethistechniquemakesdatafilebigger,butitreturnsthedesiredspeed.Alldata
whichcouldbeoverwrittenwillbewrittentotheendofthefile.
Note
When
Technical_SetTable_OverwriteIsNotAllowed
isused,
InsertPart
stilltriestoupdate
values,thatcanbringtospeedloss.Ifweneedthespeedwhileupdate,wecanusesuch
workaround
:
don'tuseInsertPart,onlyInsert
readthewholevalueintomemoryasbyte[]
thenchangeitsmiddlepart(withDBreeze.Utils.BytesProcessingCopyInsideor
CopyInsideArrayCanGrow)
insertthecompletevalue.
Allthetime
Technical_SetTable_OverwriteIsNotAllowed
canbeon.
SourcecodereceivednewfolderDBreeze\bin\Release\NET40wherewestore
DBreeze.dllreadyforMONOand.NET4>usage.ThisfolderDBreeze\bin\Release\will
holdDBreezefor.NET35(Windowsonly).
DBreezeversionfor.NET35canbeusedonlyunderWindows,causeutilizessystem
APIFlushFileBuffersfromkernel32.dll
DBreezeversionfor.NET40doesntuseanysystemAPIfunctionsandcanbeused
underLinuxMONOandunder.NET4>.ForWindows,besuretohavelatests.NET
Frameworkstartingfrom4.5,becausethereMicrosofthasfixedbugwith
FileStream.Flush(true).
[20130608]
Restoringtablefromtheothertable.
StartingfromDBreezeversion01.052wecanrestoretablefromtheothersourcetableon
thefly.
Theexamplecodeofcompaction:
privatevoidTestCompact()
{
{
tran.Commit()
}

DBreezeEngineengine2=newDBreezeEngine(@D:\temp\DBreezeTest\DBR2)
{
using(vartran2=engine2.GetTransaction())
{
//Copyingfrommainengine(Tablet1)toengine2(tablet1),withchangingallvaluesto2
foreach(varrowintran.SelectForward<int,int>("t1"))
{
tran2.Insert<int,int>(t1,row.Key,2)
}
tran2.Commit()
}
engine2.Dispose()
//engine2isfullyclosed.
//movingtablefromengine2(physicalname)tomainengine(logicalname)
tran.RestoreTableFromTheOtherFile("t1",@"D:\temp\DBreezeTest\DBR2\10000000")
//Point555
//Checking
{
foreach(varrowintran.SelectBackward<int,int>("t1"))
{
//GETTINGKEY2
Console.WriteLine("Key:{0}",row.Key)
}
}
Uptopoint555everythingwasok,whilecopyingdatafromoneengineintoanother,parallel
threadscouldreaddatafromtablet1ofthemainengine,parallelwritingthreadsofcourse
wereblockedby
command.
Startignfrompoint555someparallelthreadswhichwerereadingtablet1couldhavein
memoryreferencetotheoldphysicalfile,readingvaluesfromsuchreferencescanbringto
DBreezeTABLE_WAS_CHANGED_LINKS_ARE_NOT_ACTUALexception.
Discussionlinkis
https://dbreeze.codeplex.com/discussions/446373
Note:DONTUSECOMMITAFTER
RestoreTableFromTheOtherFileCOMMAND,justclose
transaction.
[20130613]
Fulltableslockinginsideoftransaction.
Parallelthreadscanopentransactionsandinparallelreadthesametables,inourstandard
configuration.Forwritingthreadsweusetran.SynchronizeTablescommandtosequentialize
writingthreadsaccesstothetables.
Butwhatifwewanttoblockaccesstothetableseveninparallelreadingthreads,while
modificationcommandsofourcurrenttransactionarenotyetfinished?
Forthiswehavedevelopedspecialtypeoftransaction.
using(vartran=engine.GetTransaction(eTransactionTablesLockTypes.EXCLUSIVE,"t1","p*","c$"))
{
tran.Insert<int,string>("t1",1,"Keshaisagoodparrot")
tran.Commit()
using(vartran=engine.GetTransaction(eTransactionTablesLockTypes.SHARED,"t1"))
{
tran.Insert<int,string>("t1",1,"KeshaisVERYagoodparrot")
tran.Commit()

Insideofsuchtransactionwewanttodefinethelocktypeforthelistedtables.
Note,wemustuseeitherfirsttransactiontype(engine.GetTransaction())ornewtype
(withSHARED/EXCLUSIVE)forthesametablesamongthewholeprogram.
Exampleofusage:
privatevoidExecF_003_1()
using(vartran=engine.GetTransaction(eTransactionTablesLockTypes.EXCLUSIVE,"t1","p*",
"c$"))
Console.WriteLine("T1{0}>{1}{2}",DateTime.Now.Ticks,
System.Threading.Thread.CurrentThread.ManagedThreadId,DateTime.Now.ToString("HH:mm:ss.ms"))
tran.Insert<int,string>("t1",1,"Keshaisagoodparrot")
tran.Commit()
Thread.Sleep(2000)
List<string>tbls=newList<string>()
tbls.Add("t1")
tbls.Add("v2")
using(vartran=engine.GetTransaction(eTransactionTablesLockTypes.SHARED,
tbls.ToArray()))
foreach(varrintran.SelectForward<int,string>("t1"))
Console.WriteLine(r.Value)
using(vartran=engine.GetTransaction(eTransactionTablesLockTypes.SHARED,"t1"))
//Thismustbeusedinanycase,whenSharedthreadscanhaveparallelwrites

tran.Insert<int,string>("t1",1,"KeshaisaVERYgoodparrot")
tran.Commit()
foreach(varrintran.SelectForward<int,string>("t1"))
Console.WriteLine(r.Value)
usingDBreeze.Utils.Async
privatevoidtestF_003()
Actiont2=()=>
ExecF_003_2()
t2.DoAsync()
Actiont1=()=>
ExecF_003_1()
t1.DoAsync()
Actiont3=()=>
ExecF_003_3()
t3.DoAsync()
TransactionsmarkedasSHAREDwillbeexecutedinparallel.EXCLUSIVEtransactionwill
waittillothertransactions,consumingthesametables,arestoppedandthenblockaccess
forotherthreads(readingorwriting)totheconsumingtables.
Thisapproachisgoodforavoidingtransactionexceptions,incaseofdatacompactionor
removingkeyswithfilerecreation,describedinpreviouschapter.
[20130811]
RemoveKeyValueandgetdeletedvalueandnotificationifvalueexistedinoneround.
ForthiswehaveaddedoverloadinMasterandinNestedtables:
RemoveKey
<TKey>(string
tableName,TKeykey,outboolWasRemoved,outbyte[]deletedValue)
[20130812]
InsertkeyoverloadforMasterandNestedtable,lettingnottooverwritekeyifit
alreadyexists.
ForthiswehaveaddedoverloadinMasterandinNestedtables:
publicvoid
Insert
<TKey,TValue>(stringtableName,TKeykey,TValuevalue,outbyte[]
refToInsertedValue,outbool
WasUpdated
,bool
dontUpdateIfExists
)
WasUpdatedwillbecometrue,ifvalueexists,andfalseifsuchvalueisnotinDB.
dontUpdateIfExists,
equaltotrue,willnotgiveDBtomakeanupdate.
SpeedingupselectoperationsandtraversalswithValuesLazyLoadingIsOn.
DBreezeuseslazyvalueloadingtechnique.Forexample,wecansay
varrow=transaction.Select<int,int>(t1,1)
atthismomentwereceivearow.Weknowthatsuchrowexistsbyrow.Existspropertyand
weknowitskeybyrow.Keyproperty.Atthismomentvalueisstillnottakenintomemory
fromdisk.ItwillbereadoutfromDBonlywhenweinstructrow.Value.
Sometimesitisgood,whenforustheonlykeyisenough.Suchcasescanhappenwhenwe
storesecondaryindexandthelink,totheprimarytable,asapartofthekey.Orifwehave
multiplecolumnsinonerow.Weneedtogetonlyonecolumnanddontneedtoget
complete,probablyhuge,value.
Nevertheless,lazyloadwillworkabitslower,incomparewithgettingkeyandvalueinone
round,duetoextraHDDhits.

Forthiscasewehavedevelopedintransactionaproperty/switch
tran.
ValuesLazyLoadingIsOn
.BydefaultitisON(true),justsetittofalseandall
transactiontraversalcommands,likeSelectForwards,Backwardsetc.,willreturnusrow
alreadywithareadoutValue.ThisswitchwillalsoinfluentNestedTableswhichwegetfrom
tran.InsertTable,SelectTableandrow.GetTable.
Wecansetthisswitchmanytimeswithinonetransactiontotunethespeedofdifferent
queries.
[20140603]
Storingbyte[]serializedobjectsasvalue,nativesupport.
Startingfromnowwecanbindanybyte[]serializer/deserializertoDBreezeinfollowing
manner:
ThisdeclarationmustbedonerightafterDBreezeinstantiation,beforeitsrealusage.
DBreeze.Utils.CustomSerializator.ByteArraySerializator=SerializeProtobuf
DBreeze.Utils.CustomSerializator.ByteArrayDeSerializator=DeserializeProtobuf
where...
WeusemostlyProtobuf.NETserializerinourprojects.Soexamplewillbedonealsowith
Protobuf.GetitviaNugetormakereferencetoit(protobufnet.dll).
HerearecustomwrappingfunctionsforProtobuf:
publicstaticTDeserializeProtobuf<T>(thisbyte[]data)
{
Tret=default(T)
using(System.IO.MemoryStreamms=newSystem.IO.MemoryStream(data))
{
ret=ProtoBuf.Serializer.Deserialize<T>(ms)
ms.Close()
}
returnret
}
publicstaticobjectDeserializeProtobuf(byte[]data,TypeT)
{
objectret=null
{
ret=ProtoBuf.Serializer.NonGeneric.Deserialize(T,ms)
ms.Close()
}
returnret
}
publicstaticbyte[]SerializeProtobuf(thisobjectdata)
{
byte[]bt=null
using(System.IO.MemoryStreamms=newSystem.IO.MemoryStream())
{
ProtoBuf.Serializer.NonGeneric.Serialize(ms,data)
bt=ms.ToArray()
ms.Close()
}
returnbt
}
NowletsprepareanobjectforstoringinDBreeze,decoratedwithProtobufattributes(extra
documentationaboutprotobufcanbefoundonitswebsite):
[ProtoBuf.ProtoContract]
publicclassXYZ
{
publicXYZ()
{
P1=12
P2="sdfs"
}

[ProtoBuf.ProtoMember(1,IsRequired=true)]
publicintP1{getset}
publicstringP2{getset}
}
AndnowletsuseDBreezeforstoringobject:
{
tran.Insert<int,XYZ>("t1",1,newXYZ(){P1=44,P2="well"})
tran.Commit()
}
Andforretrievingobject:
XYZobj=null
{
varrow=tran.Select<int,XYZ>("t1",1)
if(row.Exists)
{
obj=row.Value
//!!!NOTEbettertoassignrow.Valuetoobjandthenuseobjamongtheprogram.
//Callingrow.Valuecausestorereadingdatafromthetableincaseofdefault
//ValueLazyLoadingIsOn
}
}
[20160304]
ExampleofDBreezeinitializationforUWPUniversalWindowsPlatform.
stringdbr_path=System.IO.Path.Combine(Windows.Storage.ApplicationData.Current.LocalFolder.Path,
"db")
Task.Run(()=>
//System.Diagnostics.Debug.WriteLine(dbr_path)
if(engine==null)
engine=newDBreezeEngine(dbr_path)
tran.Commit()
{
varre=tran.Select<int,int>("t1",1)
System.Diagnostics.Debug.WriteLine(re.Value)
}
})
[20160320]
Quickstartquide.Customersandorders
Inthisguidewewillcreatecustomers,prototypesofbusinessordersforthesecustomers
anddeterminedifferentsearchfunctions.
Let'screateWinFormapplication,addNuGetreferencetoprotobufnetandDBreeze.Onthe
formcreateabuttonandreplacecodeoftheformwiththisone:
]
usingSystem
usingSystem.Collections.Generic
usingSystem.ComponentModel
usingSystem.Data
usingSystem.Drawing
usingSystem.Linq
usingSystem.Text
usingSystem.Threading.Tasks
usingSystem.Windows.Forms
usingDBreeze
usingDBreeze.Utils
namespaceDBreezeQuickStart
{
publicpartialclassForm1:Form
{
publicForm1()
{
InitializeComponent()
}
publicstaticDBreeze.DBreezeEngineengine=null
protectedoverridevoidOnFormClosing(FormClosingEventArgse)
{
base.OnFormClosing(e)
if(engine!=null)
engine.Dispose()
}
voidInitDb()
{
if(engine==null)
{
engine=newDBreezeEngine(newDBreezeConfiguration{DBreezeDataFolderName=
@"S:\temp\DBreezeTest\DBR1"})
//engine=newDBreezeEngine(newDBreezeConfiguration{DBreezeDataFolderName=
@"C:\temp"})
//SettingdefaultserializerforDBreeze
DBreeze.Utils.CustomSerializator.ByteArraySerializator=ProtobufSerializer.SerializeProtobuf
DBreeze.Utils.CustomSerializator.ByteArrayDeSerializator=
ProtobufSerializer.DeserializeProtobuf
}
}
publicclassCustomer
{
publiclongId{getset}
}
publicclassOrder
{
publicOrder()
{
udtCreated=DateTime.UtcNow
}
publiclongId{getset}
publiclongCustomerId{getset}
///<summary>
///Orderdatetimecreation
///</summary>
publicDateTimeudtCreated{getset}
}

///<summary>
///STARTINGTESTHERE
///</summary>
///<paramname="sender"></param>
///<paramname="e"></param>
privatevoidbutton1_Click(objectsender,EventArgse)
{
//Onetimedbinit
this.InitDb()
//Simpletest
////Testinsert
//using(vartran=engine.GetTransaction())
//{
//tran.Insert<int,int>("t1",1,1)
//tran.Insert<int,int>("t1",1,2)
//tran.Commit()
//}
////Testselect
//using(vartran=engine.GetTransaction())
//{
//varxrow=tran.Select<int,int>("t1",1)
//if(xrow.Exists)
//{
//Console.WriteLine(xrow.Key.ToString()+xrow.Value.ToString())
//}
////or
//foreach(varrowintran.SelectForward<int,int>("t1"))
//{
//Console.WriteLine(row.Value)
//}
//}
//Morecomplextest
//!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!RUNONLYONCE,THENCLEARDB
//InsertingCustomerId1
varcustomer=newCustomer(){Name="TinoZanner"}
Test_InsertCustomer(customer)
//Insertingordersforcustomer1
//for(inti=0i<5i++)
//{
//Test_InsertOrder(newOrder{CustomerId=customer.Id})
//}
//orinsertingbatchoforders
Test_InsertOrders(
Enumerable.Range(1,5)
.Select(r=>newOrder{CustomerId=customer.Id})
)
//InsertingCustomerId2
customer=newCustomer(){Name="MichaelHinze"}
Test_InsertCustomer(customer)
//Insertingordersforcustomer2
//for(inti=0i<8i++)
//{
//Test_InsertOrder(newOrder{CustomerId=customer.Id})
//}
//orinsertingbatchoforders
Test_InsertOrders(
Enumerable.Range(1,8)
.Select(r=>newOrder{CustomerId=customer.Id})
)
//Gettingallorders
Console.WriteLine("Allorders")
Test_GetOrdersByDateTime(DateTime.MinValue,DateTime.MaxValue)
//GettingOrdersofcustomer1
Console.WriteLine("Ordersofcustomer1")
Test_GetOrdersByCustomerIdAndDateTime(1,DateTime.MinValue,DateTime.MaxValue)
//GettingOrdersofcustomer2
Console.WriteLine("Ordersofcustomer2")
Test_GetOrdersByCustomerIdAndDateTime(2,DateTime.MinValue,DateTime.MaxValue)
/*
Result:
InsertedCustomerId:1,Name:TinoZanner
InsertedCustomerId:2,Name:MichaelHinze
Allorders
28.08.201507:15:57.734orderId:1
28.08.201507:15:57.740orderId:2
28.08.201507:15:57.743orderId:3
28.08.201507:15:57.743orderId:4
28.08.201507:15:57.743orderId:5
28.08.201507:15:57.757orderId:6
28.08.201507:15:57.758orderId:7
28.08.201507:15:57.758orderId:8
28.08.201507:15:57.759orderId:9
28.08.201507:15:57.759orderId:10
28.08.201507:15:57.759orderId:11
28.08.201507:15:57.760orderId:12
28.08.201507:15:57.760orderId:13
Ordersofcustomer1
28.08.201507:15:57.734orderId:1
28.08.201507:15:57.740orderId:2
28.08.201507:15:57.743orderId:3
28.08.201507:15:57.743orderId:4
28.08.201507:15:57.743orderId:5
Ordersofcustomer2
28.08.201507:15:57.757orderId:6
28.08.201507:15:57.758orderId:7
28.08.201507:15:57.758orderId:8
28.08.201507:15:57.759orderId:9
28.08.201507:15:57.759orderId:10
28.08.201507:15:57.759orderId:11
28.08.201507:15:57.760orderId:12
28.08.201507:15:57.760orderId:13
*/
return
///<summary>
///
///</summary>
///<paramname="cust"></param>
voidTest_InsertCustomer(Customercust)
{
try
{
{
//Wedon'tneedthislinebecausewewriteonlyintooneroottable.
//Addmoretablenamesforsafetransactionoperationsamongmultiple
//roottables(readdocu)
tran.SynchronizeTables("Customers")
//IntableCustomersunderkey1wewillhavenestedtablewithcustomers
vartbl=tran.InsertTable<int>("Customers",1,0)
//Underindex2wewillhavemonotonicallygrownid
if(cust.Id<1)
{
//Insert
//GettingnewIDforthecustomer
cust.Id=tran.Select<int,long>("Customers",2).Value+1
//andinsertingidbackintokey2
tran.Insert<int,long>("Customers",2,cust.Id)
}
//Insertingorupdatingofthecustomer
tbl.Insert<long,Customer>(cust.Id,cust)
//Committingentry
tran.Commit()
}
//Checkingifcustomerissaved
{
//usingSelectTableinsteadofInsertTable(readdocu).Inshortifweplantowriteand/orto
read
//fromnestedtableduringonetransactionthenweuseInsertTable,ifonlytoreadthen
SelectTable.
vartbl=tran.SelectTable<int>("Customers",1,0)
varrow=tbl.Select<long,Customer>(cust.Id)
if(row.Exists)
Console.WriteLine("InsertedCustomerId:{0},Name:{1}",row.Value.Id,
row.Value.Name)
else
Console.WriteLine("Insertfailed")
}
}
catch(Exception)
{
throw
}
///<summary>
///
///</summary>
///<paramname="order"></param>
voidTest_InsertOrder(Orderorder)
{
try
{
/*
Inourcase,wewillstoreordersofallcustomersinonetable"Orders".
Ofcoursewecouldcreateforeverycustomerhisowntable,likeOrder1,Order2...etc
Laterweareplanningtosearchorders:
1.byOrder.Id
2.byOrder.udtCreatedFromTo
3.byOrder.CustomerIdandOrder.udtCreatedFromTo
Tofulfill2and3conditionswewillneedtostoreseveralextraindicies.
*/
{
tran.SynchronizeTables("Orders")
//Underkey1wewanttostorenestedtablewithorders
vartbl=tran.InsertTable<int>("Orders",1,0)
//Underkey2wewillstoremonotonicallygrownidfororders
//Indextableforthesecondsearchconditionunderkey3
vartblDateIndex=tran.InsertTable<int>("Orders",3,0)
//Indextableforthethirdsearchconditionunderkey4
vartblCustomerAndDateIndex=tran.InsertTable<int>("Orders",4,0)
byte[]key=null
if(order.Id<1)
{
//Insert,gettingnewID
order.Id=tran.Select<int,long>("Orders",2).Value+1
//andinsertingidbackintoindex2
tran.Insert<int,long>("Orders",2,order.Id)
//InsertingsecondaryindexintotblDateIndex.
//IndexwillbecomplexandwillletussearchordersbycreationDateTime
key=
order.udtCreated.To_8_bytes_array().Concat(order.Id.To_8_bytes_array_BigEndian())
//Herewehavecompositekeydate+uniqueOrderIndex(readdocu).ValuewillbeIdof
theorderstoredintbl.
//Asavaluewecouldalsousethesameorderasintbl(redundantstorageforthehigher
speed)orpointertothekey/valueintblforSelectDirect(readdocu)
tblDateIndex.Insert<byte[],long>(key,order.Id)
//InsertingsecondaryindexintotblCustomerAndDateIndex
//KeywillstartfromCustomerId,thencomesdateTimeandthenuniqueidoftheorder
key=
order.CustomerId.To_8_bytes_array_BigEndian().ConcatMany(order.udtCreated.To_8_bytes_array(),
order.Id.To_8_bytes_array_BigEndian())
tblCustomerAndDateIndex.Insert<byte[],long>(key,order.Id)
}
//Insertingorupdatingcustomer
tbl.Insert<long,Order>(order.Id,order)
//Committingentry
tran.Commit()
}
}
catch(Exception)
{
throw
}
}
///<summary>
///
///</summary>
///<paramname="order"></param>
voidTest_InsertOrders(IEnumerable<Order>orders)
{
try
{
/*
Inourcase,wewillstoreordersofallcustomersinonetable"Orders".
Ofcoursewecouldcreateforeverycustomerhisowntable,likeOrder1,Order2...etc
Laterweareplanningtosearchorders:
1.byOrder.Id
2.byOrder.udtCreatedFromTo
3.byOrder.CustomerIdandOrder.udtCreatedFromTo
Tofulfill2and3conditionswewillneedtostoreseveralextraindicies.
*/
{
tran.SynchronizeTables("Orders")
//Underkey1wewanttostorenestedtablewithorders
vartbl=tran.InsertTable<int>("Orders",1,0)
//Underkey2wewillstoremonotonicallygrownidfororders
//Indextableforthesecondsearchconditionunderkey3
vartblDateIndex=tran.InsertTable<int>("Orders",3,0)
//Indextableforthethirdsearchconditionunderkey4
vartblCustomerAndDateIndex=tran.InsertTable<int>("Orders",4,0)
byte[]key=null

foreach(varordinorders)
{
if(ord.Id<1)
{
//Insert,gettingnewID
ord.Id=tran.Select<int,long>("Orders",2).Value+1
//andinsertingidbackintoindex2
tran.Insert<int,long>("Orders",2,ord.Id)
//InsertingsecondaryindexintotblDateIndex.
//IndexwillbecomplexandwillletussearchordersbycreationDateTime
key=
ord.udtCreated.To_8_bytes_array().Concat(ord.Id.To_8_bytes_array_BigEndian())
//Herewehavecompositekeydate+uniqueOrderIndex(readdocu).ValuewillbeIdof
theorderstoredintbl.
//Asavaluewecouldalsousethesameorderasintbl(redundantstorageforthe
higherspeed)orpointertothekey/valueintblforSelectDirect(readdocu)
tblDateIndex.Insert<byte[],long>(key,ord.Id)
//InsertingsecondaryindexintotblCustomerAndDateIndex
//KeywillstartfromCustomerId,thencomesdateTimeandthenuniqueidoftheorder
key=
ord.CustomerId.To_8_bytes_array_BigEndian().ConcatMany(ord.udtCreated.To_8_bytes_array(),
ord.Id.To_8_bytes_array_BigEndian())
tblCustomerAndDateIndex.Insert<byte[],long>(key,ord.Id)
}
//Insertingorupdatingcustomer
tbl.Insert<long,Order>(ord.Id,ord)
}
//Committingallchanges
tran.Commit()
}
}
catch(Exception)
{
throw
}
}
///<summary>
///
///</summary>
///<paramname="from"></param>
///<paramname="to"></param>
voidTest_GetOrdersByDateTime(DateTimefrom,DateTimeto)
{
try
{
{
vartbl=tran.SelectTable<int>("Orders",1,0)
vartblDateIndex=tran.SelectTable<int>("Orders",3,0)
byte[]keyFrom=
from.To_8_bytes_array().Concat(long.MinValue.To_8_bytes_array_BigEndian())
byte[]keyTo=
to.To_8_bytes_array().Concat(long.MaxValue.To_8_bytes_array_BigEndian())
foreach(varrowintblDateIndex.SelectForwardFromTo<byte[],long>(keyFrom,true,keyTo,
true))
{
varorder=tbl.Select<long,Order>(row.Value)
if(order.Exists)
Console.WriteLine(order.Value.udtCreated.ToString("dd.MM.yyyyHH:mm:ss.fff")+"
orderId:"+order.Value.Id)
}
}
}
catch(Exception)
{
throw
}
}
voidTest_GetOrdersByCustomerIdAndDateTime(longcustomerId,DateTimefrom,DateTimeto)
{
try
{
{
vartbl=tran.SelectTable<int>("Orders",1,0)
vartblCustomerAndDateIndex=tran.SelectTable<int>("Orders",4,0)
byte[]keyFrom=
customerId.To_8_bytes_array_BigEndian().ConcatMany(from.To_8_bytes_array(),
long.MinValue.To_8_bytes_array_BigEndian())
byte[]keyTo=
customerId.To_8_bytes_array_BigEndian().ConcatMany(to.To_8_bytes_array(),
long.MaxValue.To_8_bytes_array_BigEndian())
foreach(varrowintblCustomerAndDateIndex.SelectForwardFromTo<byte[],
long>(keyFrom,true,keyTo,true))
{
varorder=tbl.Select<long,Order>(row.Value)
if(order.Exists)
Console.WriteLine(order.Value.udtCreated.ToString("dd.MM.yyyyHH:mm:ss.fff")+"
orderId:"+order.Value.Id)
}
}
}
catch(Exception)
{
throw
}
}
publicstaticclassProtobufSerializer
{
///<summary>
///Deserializesprotobufobjectfrombyte[]
///</summary>
///<typeparamname="T"></typeparam>
///<paramname="data"></param>
///<returns></returns>
publicstaticTDeserializeProtobuf<T>(thisbyte[]data)
{
Tret=default(T)
{
ret=ProtoBuf.Serializer.Deserialize<T>(ms)
ms.Close()
}
returnret
}
///<summary>
///Deserializesprotobufobjectfrombyte[].Nongenericstyle.
///</summary>
///<paramname="T"></param>
publicstaticobjectDeserializeProtobuf(byte[]data,TypeT)
{
objectret=null
{
ret=ProtoBuf.Serializer.NonGeneric.Deserialize(T,ms)
ms.Close()
}

returnret
}
///<summary>
///Serializeobjectusingprotobufserializer
///</summary>
publicstaticbyte[]SerializeProtobuf(thisobjectdata)
{
byte[]bt=null
using(System.IO.MemoryStreamms=newSystem.IO.MemoryStream())
{
ProtoBuf.Serializer.NonGeneric.Serialize(ms,data)
bt=ms.ToArray()
ms.Close()
}
returnbt
}
}
}
[20160329]
DBreeze.DataStructures.DataAsTree
DuetothedesireofsomepeopletoimplementintoDBreezeanabilitytostoredataasa
tree,withdependentnodes,outofthebox,wehavecreatednewnamespace
DBreeze.DataStructures.AndinsidethereisaclassDataAsTree.
Howtoworkwiththat:
usingDBreeze
usingDBreeze.DataStructures
DataAsTreerootNode=null
DataAsTreeinsertedNode=null
//Inthis"testtree"wewillstoreournewDataStructure,soitshouldbesynchronizedwithother
tables,
//ifwewanttomodifyit
there)
tran.SynchronizeTables("testtree")
//Initializingrootnode.Mustbeinitializedafteranynewtransaction(ifDataAsTreemustbeused
rootNode=newDataAsTree("testtree",tran)
//Addingtotherootnodeasinglechildnode
rootNode.AddNode(newDataAsTree("folder1"))
//Insertingsecondchildnode,gettingreferencetoinsertednode
insertedNode=rootNode.AddNode(newDataAsTree("folder2"))
//Preparinganodebatch
varnodes=newList<DataAsTree>()
nodes.Add(newDataAsTree("xfolder1"))
nodes.Add(newDataAsTree("xfolder2"))
//nodes.Add(newDataAsTree("xfolder2"))
nodes.Add(newDataAsTree("xfile1"))
//Andinsertingitunderthesecondrootchildnode
insertedNode.AddNodes(nodes)
//Insertingnodewiththecontent(itcanbecountedasfile,thouanynodecanhaveContent)
varfileNode=newDataAsTree("file1")
fileNode.NodeContent=newbyte[]{1,2,3,4,5}
//Addingitalsototheroot
rootNode.AddNode(fileNode)
//Committingtransaction,soallourchangesaresavednow
tran.Commit()
}//eousing
Ok,nowletsiteratethroughnodes
//Againcreatingrootnode(alwayswhenwestartnewtransactionitmustbeperformed)
//AndrecursivelyreadallourinsertednodesstartingfromRoot(anynodecanbeused)
foreach(vartninrootNode.ReadOutAllChildrenNodesFromCurrentRecursively(tran))
Console.WriteLine(tn.NodeName+"_"+tn.NodeId+"_"+tn.ParentNodeId)
byte[]cnt=tn.GetContent(tran)
if(cnt!=null)
//Showingcontentofthefile
}
}
}//eousing
Now,letsgrabnodesbyspecifiednameandrebindthemtootherparentchangethem:
//Reconnectingallnodesfrom2parentIdto1parentId
foreach(vartninrootNode.GetNodesByName("xf"))//or
rootNode.GetFirstLevelChildrenNodesByParentId(2)
rootNode.RemoveNode(tn)
tn.ParentNodeId=1
rootNode.AddNode(tn)
tran.Commit()
}//eousing
Now,letsrenamenodesandsupplydifferentcontent
//Renamingnodesandsettingnewcontent
foreach(vartninrootNode.GetNodesByName("xf"))
{
tn.NodeName=tn.NodeName+"_new_"
tn.NodeContent=newbyte[]{7,7,7}
rootNode.AddNode(tn)
}
tran.Commit()
}//eousing
Ifsomethingisnotworkinglikeitisexpected,please,donthesitatetowritedownanissue
commenton
http://dbreeze.tiesky.com
Copyright2012dbreeze.tiesky.com/AlexeySolovyov/IvarsSudmalis

DBreeze Documentation 032

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

DBreeze Documentation 032

Uploaded by

Copyright:

Available Formats

DBreezeDatabaseDocumentation.

You might also like