26/03/14

Making Django ORM more DRY with prefixes and Qs | hackerluddite

hackerluddite
Not mere sabotage

MakingDjangoORMmoreDRYwithprefixesandQs
PostedonJuly7,2012

This post builds on Jamie Mathews excellent Building a higher-level query API: the right way to use Djangos ORM, which makes the solid argument that the Manager API is a Lie. If you havent read that post, head over that way to hear why its a fantastic idea to build high level interfaces to your models, and because of limitations in Djangos manager API, to do so in QuerySets rather than Managers. This post tackles the problem: how do we get a high level interface across relationships? Suppose we have the following models:

f r o m d j c l a s s M u s e o r g s t a e n d c l a s s

a n g o e m b e r = a n i z r t = = m

. d b r s h m o d a t i m o o d e

i m i p ( e l s o n d e l l s .

p o r t m o d e . F o r = m o s . D a D a t e

m o l s . e i g d e l t e T T i m

d e l M o d n K e s . F i m e e F i

s e l ) y ( U o r e F i e e l d : s e r i g n l d ( ( n u ) l l = T r u e ) ) K e y ( O r g a n i z a t i o n )

O r g a n i z a t i o n ( m o d e l s . M o d e l ) : n a m e = m o d e l s . C h a r F i e l d ( m a x _ l e n g t h = 2 5 )

The high-level notion wed like to get at with these models is whether a membership is current. Suppose the definition of a current membership is: A membership is current if now is after the start date, and either the end date is null (a perpetual membership), or now is before the end date (not yet expired). This is Django ORM 101 here: in order to get that or logic, we either need to combine two querysets with `|`, or to do the same with Q objects. For reasons that will become obvious below, Ill go the Q route. As in Jamie Mathews post, well use the PassThroughManager from django-model-utils to get our logic into both the QuerySets and the Manager.

f r o f r o f r o f r o m m m

d j a d j a d a t m o d

n g o n g o e t i e l _

. d b . d b . m e i u t i l e r s h r r e n w = t u r n Q (

i m p m o d m p o s . m i p Q t ( s d a t s e e n d

o r t e l s r t d a n a g u e r e l f e t i l f . _ _ i y S e ) : m e . f i l s n u d e l o r e F o r D a t

m o d e l i m p o r a t e t i e r s i t

s Q m e m p o r t P a s s T h r o u g h M a n a g e r

c l a s s d e f

M e m b c u n o r e

t ( m o d e l s . q u e r y . Q u e r y S e t ) : n o w ( ) t e r ( s t a r t _ _ l t e = n o w , l l = T r u e ) | Q ( e n d _ _ g t e = n o w ) ) s . M i g n e i g e T i o d e K e y n K e m e F l ) : ( U s e r ) y ( O r g a n i z a t i o n ) i e l d ( )

c l a s s

M e m b e r s u s e r = m o g r o u p = m s t a r t = m

h i p d e l o d e o d e

( m o s . F l s . l s .

https://hackerluddite.wordpress.com/2012/07/07/making-django-orm-more-dry-with-prefixes-and-qs/

1/3

26/03/14

Making Django ORM more DRY with prefixes and Qs | hackerluddite

e n d

m o d e l s . D a t e T i m e F i e l d ( n u l l = T r u e ) = P a s s T h r o u g h M a n a g e r . f o r _ q u e r y s e t _ c l a s s ( M e m b e r s h i p Q u e r y S e t ) ( )

o b j e c t s c l a s s

O r g a n i z a t i o n ( m o d e l s . M o d e l ) : n a m e = m o d e l s . C h a r F i e l d ( m a x _ l e n g t h = 2 5 )

This works well we can now get current memberships with the high-level, conceptually friendly:

> > >M e m b e r s h i p . o b j e c t s . c u r r e n t ( )

But suppose we want to retrieve all the Organizations which have current members? Whether using a Manager class or QuerySet class to define our filtering logic, were stuck: the notion of current is baked into the QuerySet (or manager) of the original class. If we come from a related class, we have to repeat the logic, prefixing all of the keys:

> > > . . .

O r g a n i z a t i o n . o b j e c t s . f i l t e r ( m e m b e r s h i p _ _ s t a r t _ _ l t e = n o w , Q ( m e m b e r s h i p _ _ e n d _ _ i s n u l l = T r u e ) | Q ( m e m b e r s h i p _ _ e n d _ _ g t e = n o w ) )

This breaks DRY if we ever need to change the logic for current (say, to add `dues_payed=True`), we have to find all the instances and fix it. Bug magnet!

Prefixed Q Objects
Heres one possible solution to this problem. The idea is to build the logic for a query using a custom Q class, which dynamically prefixes its arguments:

f r o m f r o m c l a s s

d j a n g o . d b i m p o r t d j a n g o . d b . m o d e l s P r e f a c c e s s d e f _ _ # s u ) ) d e f i x e o r i n i P r e p e r d Q ( Q = " " t _ _ ( f i x ( P r e ( s e l f ) : s e l f , a l l t h f i x e d Q . p r e f i

m o d e l s i m p o r t Q

* * k w e d i , s e x ( k )

a r g s ) : c t i o n a r y k e y s l f ) . _ _ i n i t _ _ ( * * d i c t ( , v ) f o r k , v i n k w a r g s . i t e m s ( )

p r e f i x ( s e l f , * a r g s ) : r e t u r n " _ _ " . j o i n ( a f o r a

i n

( s e l f . a c c e s s o r , )

a r g s

i f

a )

c l a s s

M e m b e r s h i p Q u e r y S e t ( m o d e l s . q u e r y . Q u e r y S e t ) : c l a s s M Q ( P r e f i x e d Q ) : # " m e m b e r s h i p " Q a c c e s s o r = " " # U s e a n e m p t y a c c e s s o r d e f m e m b e r s h n o w = d a r e t u r n s s e l f ) d e f i p _ t e t e l f . M Q c u r i m e . M Q ( e n r e n . n o ( s t d _ _

- -

n o

p r e f i x .

t _ q ( s e l f ) : w ( ) a r t _ _ l t e = n o w ) & ( i s n u l l = T r u e ) | s e l f . M Q ( e n d _ _ g t e = n o w )

c u r r e n t ( s e l f ) : r e t u r n s e l f . f i l t e r ( s e l f . m e m b e r s h i p _ c u r r e n t _ q ( ) )

https://hackerluddite.wordpress.com/2012/07/07/making-django-orm-more-dry-with-prefixes-and-qs/

2/3

26/03/14

Making Django ORM more DRY with prefixes and Qs | hackerluddite

Now that weve abstracted the definition of current into the prefixed-Q class, we can subclass this QuerySet, and override the prefix in our related class:

c l a s s #

O r g a n o v e r r c l a s s M a c c d e f

i z a t i o n Q u i d e t h e s Q ( P r e f i x e e s s o r = "

e r y u p e d Q ) m e m :

S e t ( M e m b e r s h i p Q u e r y S e t ) : r c l a s s ' s M Q d e f i n i t i o n , t o b e r s h i p "

a d d

o u r

p r e f i x :

w i t h _ c u r r e n t _ m e m b e r s ( s e l f ) : r e t u r n s e l f . f i l t e r ( s e l f . m e m b e r s h i p _ c u r r e n t _ q ( ) )

> > > #

O r g a n i z a t i o n . o b j e c t s . w i t h _ c u r r e n t _ m e m b e r s ( ) s h o u l d d o t h e r i g h t t h i n g !

This trick will work with multiple different relations youll just need to use a different Q subclass name for each relation, so they dont conflict, and mix in super classes:

c l a s s

O r g a n c l a s s M a c c c l a s s E a c c

i z a Q ( P e s s Q ( P e s s

t i o r e f o r r e f o r

n Q u e i x e d = " m i x e d = " e

r y S e t ( M e m b e r s h i p Q u e r y S e t , Q ) : e m b e r s h i p " Q ) : x e c u t i v e s "

E x e c u t i v e Q u e r y S e t ) :

While this definitely has the feel of slightly noodley trickery to it, I think it starts to get at the core of what we might want in a more robust future ORM for Django: the ability to define high-level logic to assign meaning to particular collections of fields, and to preserve that logic across relations and chains. Does this meet a use case you have?

Share this:

Twitter

Facebook

Like
Bethefirsttolikethis.

Thisentrywaspostedindjango.Bookmarkthepermalink.

hackerluddite
The Twenty Ten Theme Blog at WordPress.com.

Follow

Followhackerluddite
Geteverynewpostdelivered toyourInbox. Enteryouremailaddress
Signmeup

PoweredbyWordPress.com

https://hackerluddite.wordpress.com/2012/07/07/making-django-orm-more-dry-with-prefixes-and-qs/

3/3