Is NoSQL for me? I’m just a small fish

Many big players are using NoSQL as their storage mechanism. When reading or hearing about NoSQL, names such as Google, Facebook or Amazon are brought up. And why not? I mean surely if they’ve gone down the road of NoSQL and things are working out for them, despite the occasional hiccup, it must be good. It provides credibility.

What can often be viewed as a major “selling” point of NoSQL however, can also do it disservice to an extent. See, when people hear about success stories like this, they sometimes discard the NoSQL option, thinking that they’re just a small shop doing typical Line of Business applications for their customers. They don’t have the users or demands that companies such as Facebook have. As such, they stay within their comfort zone of relational systems, bearing the pains. That doesn’t necessarily have to be the case. NoSQL is viable for small, medium and large fish.

I didn’t have scaling issues, I had maintainability nightmares.

I’ve been writing LOB applications for as long as I can remember. With regard to data access I’ve gone from Clipper, to embedded SQL, to Drag and Drop, hand-rolled ORM’s and OSS/Commercial ORM’s.

Lately, with ORM’s, each time I started a new application, I would dread having to set up mapping of entities to tables. Granted that many solutions out there helped (AutoPersistenceModel from FluentNH helped a lot!), but it still bugged me. There were always edge-cases. There were always some things that just couldn’t be done easily.

And then of course the migrations! And I’m not talking about migrations during development, but deployment. Alter table, Drop Column. No. OK. Alter table, Add Column, Copy Data. Drop Column. No. Crap. Alter table. Drop Constraint. No. I’m sure we’ve all been there, and we’ve all solved it one way or another, and with certain success, but still it bugged me.

I wanted to look for something different, something that would allow me to seriously not have to worry so much about these things. Something I could just stuff some data in to and get it out. That’s what made me look at document databases, and in particular CouchDB. The main thing that attracted me to NoSQL, wasn’t that it was scalable, but that it was schema-less*. I could store whatever I wanted. I didn’t have to worry about violating some constraint**. It was the Mapping and Migrations I wanted to get rid of. Removing many of the issues associated with mappings and migrations, maintenance has become simpler, which in turn has provided me with more flexibility in design.

But I’ve also obtained other advantages. CouchDB is based on HTTP and embraces it as an application protocol (we could say it complies with Level 2 of Richardson’s Maturity Model). It uses HTTP status codes, cache infrastructure and other benefits of the web infrastructure that is already proven and in place to offer power and simplicity in design. As someone that loves HTTP, for me this has been a win-win situation.

It has also opened up the door for me to write disconnected applications. By providing support for numerous platforms (including Android) and it’s powerful master-master replication mechanism, it allows me to easily write offline applications that can sync when connected.

CouchDB offers me authentication, authorization, validation. In fact based usage scenarios, you don’t even need to use a client to access CouchDB. Many still don’t realize but CouchDB is a fully-fledged application server. You can create views, listing, data input, all with the same server.

So what started out as just a reason to remove mappings has proven much more fruitful.

Persistence Ignorance has shifted

Although it might seem so at first, it’s not all a bed of roses however. I always mention in my presentations on CouchDB, that as much as we strived to accomplish Persistence Ignorance when working with ORM’s, when it comes to using a NoSQL solution, we can’t be ignorant about our persistence mechanism, from an architectural perspective. Depending on what we need, how we need to store the information, and how we need to query it, are factors that play an important role in the solution we opt for. This applies not only to when choosing the NoSQL solution, but also when persisting information. It is true that joins and complex queries have disappeared in Document databases, but relationships between objects in the real world have not, and based on how we need to work with this information, different techniques can should be applied.

Some of the benefits I’ve outlined here are specific to CouchDB. Other NoSQL solutions offer different advantages. It depends on what you’re looking for is where you’d pick one or another.  However, don’t be fooled into thinking that the only thing NoSQL has to offer is scalability. There’s a whole lot more that you can gain.

As an added bonus of course, when your application does hit the big time, it might even scale.

 

* When I talk about schema-less here I'm referring to the rigid column structure that traditional relational storage systems enforce us into following (i.e. defining columns beforehand). Most document databases including CouchDB and RavenDB do in fact store schema with each document. By using JSON, each document is self-describing, so technically there is a schema. It's just not "fixed". 

** For all those obsessed with imposing constraints on the storage mechanism, you can do this with CouchDB if you want.


Posted 11-24-2010 10:40 AM by Hadi Hariri

[Advertisement]

Comments

Paul Cowan wrote re: Is NoSQL for me? I’m just a small fish
on 11-25-2010 5:53 AM

Do you not think the next logical step is for the big database vendors to start some of the nosql goodness into their products?

This should force them to evolve.

Maybe not, of course

Martin wrote re: Is NoSQL for me? I’m just a small fish
on 11-25-2010 5:35 PM

You mention that complex queries have disappeared in NoSQL, admitted that real world objects still have complex relationships and just mentioned that different techniques need to be used to address this situation.  I have not yet seen a post that describes what those techniques might be.  I assume that you would write a lot of Map Reduces to make the queries simpler, but is this the only technique?

Matt Freeman wrote re: Is NoSQL for me? I’m just a small fish
on 11-26-2010 1:31 AM

Paul, RDBMS can only evolve in the context they exist in, you can still do hybrid NoSQL over a SQL store but its messy, think blobs and managing your indexes.  RDBMS will have their place but I dont want them to come with nosql with all their baggage, new products fine..

A friend of mine, recently said "Matt, if this was the way forward dont you think Microsoft would of done it?"  He is now starting to see the light as a massively complexed query in t-SQL was reduced to a 3 line couchdb view.

I am using both CouchDB and RavenDB in two live products now, along with some legacy stuff using SQL Server. Any greenfield project is now on CouchDB as a rule of thumb, not to be cool, as this blog say to remove the mapping and migration headache is a bonus, but for me its precomputed indexes (with aggregates - before you mention indexed views) that are awesome.

Dean wrote re: Is NoSQL for me? I’m just a small fish
on 11-26-2010 5:25 AM

Good article.  As an RDBMS developer with an open mind, I find this debate very interesting.

I would make a couple of observations regarding this article.  Firstly, it seems to emphasise just one of the many considerations to be taken into account in making the decision between relational and NoSQL - ease of development and deployment.  This is actually a fair way down my list of considerations.  If I'm building an application that I hope to last for maybe 10 years, maybe even longer, then a few extra days of development time would not prevent me from using the right tool for the job.  Of course this article does mention the other considerations - scalability and so on - but the main gripe with RDBMS seems to be ease of development/deployment.

Secondly, and this is in common with almost all articles on NoSQL, it seems to regard the database layer as simply a way of persisting data to be used in the application.  I have always been of the opinion that a database should be properly designed to store the organisation's information - independently of any applications that might access it.  Maybe this is old fashioned thinking nowadays, I don't know.

Either way, I am hoping to be able to work with NoSQL in the near future to get a good luck at the alternatives in action.

Hadi Hariri wrote re: Is NoSQL for me? I’m just a small fish
on 11-26-2010 7:02 AM

@Martin,

It depends on what you want to do and most importantly, what system you are using. Take an invoice for instance, that has details. If you store the details in the same document, for querying it's simple. However, updating means you'll need to update entire document, including details (I'm talking CouchDB here which currently doesn't have PATCH support). This might be OK for an invoice since you regularly don't update invoices, and if you do, most likely other details change.

However, for a blog with comments it might not be. You don't want to persist the entire blog and ALL comments on each new comment. So you'd store the blog as one document and the comments as another.

So it very much depends on the system you use and scenarios.

Hadi Hariri wrote re: Is NoSQL for me? I’m just a small fish
on 11-26-2010 7:05 AM

@Dean,

Thanks. Yes, I highlight two main points which were reasons for me to look elsewhere and based on my needs for specific situations. My point was that it's not only about scalability.

Regarding a long running application, when you're deploying it to 100 customers each on different versions, you can to an extend automate upgrades of DB's (otherwise it's really unmanagable), and I've done so quite successfully in my time. However, there are always problems.

Regarding the storage serving as documentation of the data, as I mentioned, JSON is self-describing so you do not lose that (at least with NoSQL solutions that take the approach to use JSON or something similar).

Matt Freeman wrote re: Is NoSQL for me? I’m just a small fish
on 11-28-2010 1:45 AM

CouchDB supports Update Handlers which mean you can update only small portions of a doc if you want and do some pretty funk stuff, IMHO it works much better than PATCH operations. In all circumstances where Ive used PATCH (.e.g against RavenDB) the solution is much nicer with CouchDB update handlers.  I dont see the need to support PATCH now in CouchDB except to satisfy REST purests

Martin wrote re: Is NoSQL for me? I’m just a small fish
on 12-06-2010 7:15 PM

Me again.  So, I've had a chance to look at CouchDB now and it seems to me that it would be a great data store but only for a limited number of applications.  

When it comes to providing something similar to joins in SQL, the map functions seem to only allow limited complexity.  I mean parent/child relations are fairly simple, but that's about as complex as it seems to go.

And, even if you worked in that complexity for every join type operation you needed, for an application with even mediocre data complexity you would probably stumble across a performance issue with write operations due to the nature of the way views are implemented.  Views are implemented in such a way that every new document or revision needs to be processed by every view.  The more complex your data, the more views you need, the slower the writes.

I can see why every example that I have seen uses a blog as the target application - minimal data complexity, insignificant number of writes to reads (unless it is a boring blog ;-)), low priority on data immediacy.

I might be wrong.  What is your take?

Thanks

Martin

Hadi Hariri wrote re: Is NoSQL for me? I’m just a small fish
on 12-10-2010 6:28 AM

@Martin,

It's hard to get into detail without knowing exact scenarios, but what I have found over the years that often the problem doesn't stem from the limitations of the frameworks or tools, but the way we are modelling our systems and architectures.

Most examples you see out there do in fact use Blogs/Comments as examples but mostly because it's a domain that many are familiar with.  

devlicio.us wrote re: Is NoSQL for me? I’m just a small fish
on 04-02-2011 7:31 PM

Is nosql for me i m just a small fish.. Amazing :)

devlicio.us wrote re: Is NoSQL for me? I’m just a small fish
on 04-22-2011 8:42 AM

Is nosql for me i m just a small fish.. Bang-up :)

devlicio.us wrote re: Is NoSQL for me? I’m just a small fish
on 06-02-2011 7:33 PM

Is nosql for me i m just a small fish.. Amazing :)

devlicio.us wrote re: Is NoSQL for me? I’m just a small fish
on 06-04-2011 7:54 PM

Is nosql for me i m just a small fish.. Nice :)

devlicio.us wrote re: Is NoSQL for me? I’m just a small fish
on 06-12-2011 4:41 AM

Is nosql for me i m just a small fish.. Amazing :)

ds16 porn u3ne wrote re: Is NoSQL for me? I’m just a small fish
on 07-02-2011 1:38 AM

Is nosql for me i m just a small fish.. Nice :)

ales porn hjcf wrote re: Is NoSQL for me? I’m just a small fish
on 07-02-2011 6:59 PM

Is nosql for me i m just a small fish.. Not so bad :)

pbtlqtpyyz wrote re: Is NoSQL for me? I’m just a small fish
on 08-30-2011 12:16 PM
mgqhoyzvci wrote re: Is NoSQL for me? I’m just a small fish
on 08-30-2011 3:22 PM

gqhybefwmjdjpvt, http://www.fixaclckiw.com phmlchvasd

cvosjwwybc wrote re: Is NoSQL for me? I’m just a small fish
on 08-30-2011 6:03 PM

qpqduefwmjdjpvt, http://www.nnkuqeevcy.com awnhdgqgmz

bookmarking services wrote re: Is NoSQL for me? I’m just a small fish
on 01-19-2013 1:03 AM

A9fP13 Hey, thanks for the blog article.Thanks Again.

lose weight pills wrote re: Is NoSQL for me? I’m just a small fish
on 01-31-2013 4:54 PM

16xiZD I really like and appreciate your blog post.Really looking forward to read more. Really Cool.

buy imitrex wrote re: Is NoSQL for me? I’m just a small fish
on 02-14-2013 10:49 AM

kwbsrM Thank you ever so for you post.Much thanks again. Cool.

buy clomid wrote re: Is NoSQL for me? I’m just a small fish
on 03-01-2013 9:02 AM

wz4RSb I cannot thank you enough for the blog.Much thanks again. Awesome.

Social bookmarks wrote re: Is NoSQL for me? I’m just a small fish
on 03-25-2013 2:39 AM

MQWZ9i I appreciate you sharing this blog article.Really looking forward to read more. Cool.

best compact digital cameras wrote re: Is NoSQL for me? I’m just a small fish
on 05-14-2013 4:01 AM

hvvUqB Great, thanks for sharing this blog. Great.

best social bookmarks wrote re: Is NoSQL for me? I’m just a small fish
on 06-19-2013 7:08 AM

aalOmh I think this is a real great article. Keep writing.

great service wrote re: Is NoSQL for me? I’m just a small fish
on 07-04-2013 1:47 PM

6Oujn2 Major thankies for the blog post. Want more.

news wrote re: Is NoSQL for me? I’m just a small fish
on 07-26-2013 8:17 AM

N5bhbl I loved your blog. Really Cool.

amazing news wrote re: Is NoSQL for me? I’m just a small fish
on 08-02-2013 4:01 PM

E0QpD6 A round of applause for your blog.Really thank you! Cool.

best news of them all wrote re: Is NoSQL for me? I’m just a small fish
on 08-04-2013 4:05 PM

xgEEjo Im obliged for the article post.Really looking forward to read more. Much obliged.

great link buildng wrote re: Is NoSQL for me? I’m just a small fish
on 08-19-2013 8:54 PM

d9UWpQ Thanks a lot for the blog.Really thank you! Really Cool.

great seo service wrote re: Is NoSQL for me? I’m just a small fish
on 09-04-2013 12:09 AM

LQ6zSm Hey, thanks for the blog post.Much thanks again. Fantastic.

make money online wrote re: Is NoSQL for me? I’m just a small fish
on 09-11-2013 7:13 PM

jMM8w5 Thank you ever so for you article.Really looking forward to read more. Fantastic.

awesome link building wrote re: Is NoSQL for me? I’m just a small fish
on 09-24-2013 3:03 PM

pZJgjj Thanks for sharing, this is a fantastic article post.Really looking forward to read more. Much obliged.

check out these guys! wrote re: Is NoSQL for me? I’m just a small fish
on 10-15-2013 7:56 PM

wONqi7 Thanks a lot for the blog article.Much thanks again. Will read on...

take a look at it! wrote re: Is NoSQL for me? I’m just a small fish
on 10-25-2013 4:13 AM

2iO1cE Say, you got a nice article post.Really looking forward to read more. Want more.

smashing top seo wrote re: Is NoSQL for me? I’m just a small fish
on 11-01-2013 9:19 AM

3luACd I value the article.Really looking forward to read more. Fantastic.

high pr backlinks wrote re: Is NoSQL for me? I’m just a small fish
on 07-16-2014 7:25 PM

Xyjk5T Awesome article post.Thanks Again. Want more.

high quality backlinks wrote re: Is NoSQL for me? I’m just a small fish
on 07-18-2014 11:13 AM

CGNoTH I value the post.Much thanks again. Really Great.

Add a Comment

(required)  
(optional)
(required)  
Remember Me?

About The CodeBetter.Com Blog Network
CodeBetter.Com FAQ

Our Mission

Advertisers should contact Brendan

Subscribe
Google Reader or Homepage

del.icio.us CodeBetter.com Latest Items
Add to My Yahoo!
Subscribe with Bloglines
Subscribe in NewsGator Online
Subscribe with myFeedster
Add to My AOL
Furl CodeBetter.com Latest Items
Subscribe in Rojo

Member Projects
DimeCasts.Net - Derik Whittaker

Friends of Devlicio.us
Red-Gate Tools For SQL and .NET

NDepend

SlickEdit
 
SmartInspect .NET Logging
NGEDIT: ViEmu and Codekana
LiteAccounting.Com
DevExpress
Fixx
NHibernate Profiler
Unfuddle
Balsamiq Mockups
Scrumy
JetBrains - ReSharper
Umbraco
NServiceBus
RavenDb
Web Sequence Diagrams
Ducksboard<-- NEW Friend!

 



Site Copyright © 2007 CodeBetter.Com
Content Copyright Individual Bloggers

 

Community Server (Commercial Edition)