Computer Science Canada

Storing binary files with DataMapper

Author:  DtY [ Wed Aug 19, 2009 10:51 am ]
Post subject:  Storing binary files with DataMapper

What is the best way to store binary files in a DataMapper database? They wont be too big, just images in three sizes, the biggest is 300x300 (jpeg).
I tried looking for an extension to do this specifically, but found nothing.
I was thinking it might work to store an RMagick marshalled object, since I will only be directly using the images with ImageMagick. (If storing the actual file it needs to be able to directly load out of the database, I assume RM has a method to load an image from a string?)
I'm going to deploy it with Heroku, so I don't have access to a filesystem.

Author:  Tony [ Wed Aug 19, 2009 11:05 am ]
Post subject:  RE:Storing binary files with DataMapper

Are you sure you want to be storing images in a database? External filesystems (such as S3) are a popular choice, though I'm not familiar with what tools you'd have available beyond attachment_fu (RoR plugin), which I assume is not applicable.

Author:  DtY [ Wed Aug 19, 2009 11:25 am ]
Post subject:  RE:Storing binary files with DataMapper

I'd prefer not to use external storage, I'd like to be able to work on it locally when I don't have a connection. I guess it is an option though. I don't want to pay for storage though, the project is mainly for my personal use.
I don't think using remote storage would be any faster than using a postgreSQL database. The output will be rather static, so heroku will be caching it, so pulling the images doesn't need to be insanely fast.

I dunno if it will help at all to know, but I'm working on a last.fm album collage type thing like this, except skipping albums without album art, and choosing different sizes.

Author:  gianni [ Wed Aug 19, 2009 11:34 am ]
Post subject:  RE:Storing binary files with DataMapper

Storing images (or virtually any binary file) in a database is frowned upon. There are very few legitimate use-cases where it would be acceptable.

Perhaps you can upgrade your Heroku plan, or use a different server/host?

Author:  DtY [ Wed Aug 19, 2009 11:52 am ]
Post subject:  Re: RE:Storing binary files with DataMapper

gianni @ Wed Aug 19, 2009 11:34 am wrote:
Storing images (or virtually any binary file) in a database is frowned upon. There are very few legitimate use-cases where it would be acceptable.

Perhaps you can upgrade your Heroku plan, or use a different server/host?

Afaik, no Heroku plan allows filesystem access, it's part of their clustering system or something.
Are there any other free ruby hosting plans that will let me run a rack application?

[edit] I realized every other piece of this program would require an internet connection to do anything, so maybe external storage might work

[edit 2] Actually, it might be worth reloading the image from last.fm each time it's generated. Heroku will cache each final image, so I don't think grabbing the images from last.fm will be too much bandwidth.
I don't know if last.fm would like that though.. they request you cache stuff so you aren't making a lot of requests. Heroku doesn't cache outgoing http requests, does it?

Author:  rizzix [ Wed Aug 19, 2009 2:21 pm ]
Post subject:  RE:Storing binary files with DataMapper

I don't know if this as been mentioned, but you can just: base64encode(deflate(image))

Then store the result as a CLOB in the database.

Just inflate(base64decode(clob)) to get the image back.

Note: I'm not sure what the exact function names are in Ruby.

Author:  DtY [ Wed Aug 19, 2009 2:45 pm ]
Post subject:  Re: RE:Storing binary files with DataMapper

rizzix @ Wed Aug 19, 2009 2:21 pm wrote:
I don't know if this as been mentioned, but you can just: base64encode(deflate(image))

Then store the result as a CLOB in the database.

Just inflate(base64decode(clob)) to get the image back.

Note: I'm not sure what the exact function names are in Ruby.

I never thought of Base64, thanks!

Author:  Tony [ Wed Aug 19, 2009 3:22 pm ]
Post subject:  RE:Storing binary files with DataMapper

If you must, Base64 is the simplest way to go.

The problem is that ultimately you'd be unpacking the data from the DB and creating a new image file (on disk) in a temp directory anyway. And since it will be a temporary file, it must be marked as "do-not-cache" for client-side.


: