Sunday, August 12, 2007

Upload

upload_to_s3 - Ruby S3 upload client

Prerequisites:
gem install aws-s3
gem install main

#!/bin/env ruby  require 'rubygems' require 'main' require 'aws/s3' include AWS::S3  Main {   argument('source_filename') {     cast :string     description 'source filename to copy to S3'   }    argument('bucket_name') {     cast :string     description 'bucket to place the file in on S3'   }    option('access_key_id') {     argument :optional     description 'specify the access_key_id manually'     default 'put your access key here if you want'   }    option('secret_access_key') {     argument :optional     description 'specify the secret key manually'     default 'put your secret key here if you want'   }    def run     bucket_name = params['bucket_name'].value     source_filename = params['source_filename'].value      Base.establish_connection!(       :access_key_id     => params['access_key_id'].value,       :secret_access_key => params['secret_access_key'].value     )      begin       Bucket.find(bucket_name)     rescue       puts "Need to make bucket #{bucket_name}.."       Bucket.create(bucket_name)        # Confirm its existence..       Bucket.find(bucket_name)     end      puts "Got bucket.."     puts "Uploading #{File.basename(source_filename)}.."     S3Object.store(File.basename(source_filename), open(source_filename), bucket_name)     puts "Stored!"      exit_success!   end }

Make a remote URL work like a file upload (in Rails)

Want to load a remote URL into an acts_as_attachment/attachment_fu model? Use this little utility class.

  class UrlUpload     EXTENSIONS = {       "image/jpeg" => ["jpg", "jpeg", "jpe"],       "image/gif" => ["gif"],       "image/png" => ["png"]     }     attr_reader :original_filename, :attachment_data     def initialize(url)       @attachment_data = open(url)       @original_filename = determine_filename     end      # Pass things like size, content_type, path on to the downloaded file     def method_missing(symbol, *args)       if self.attachment_data.respond_to? symbol         self.attachment_data.send symbol, *args       else         super       end     end          private       def determine_filename         # Grab the path - even though it could be a script and not an actual file         path = self.attachment_data.base_uri.path         # Get the filename from the path, make it lowercase to handle those         # crazy Win32 servers with all-caps extensions         filename = File.basename(path).downcase         # If the file extension doesn't match the content type, add it to the end, changing any existing .'s to _         filename = [filename.gsub(/\./, "_"), EXTENSIONS[self.content_type].first].join(".") unless EXTENSIONS[self.content_type].any? {|ext| filename.ends_with?("." + ext) }         # Return the result         filename       end   end 


Now when you have the URL you want to load, do something like this:

@model.uploaded_data = UrlUpload.new(url) 


Or better yet, make a pseudo-accessor on your aaa/attachment_fu model so you can stay "model-heavy".

def url=(value)   self.uploaded_data = UrlUpload.new(value) end 

Very simple php file upload

I think this is the minimum necessary to upload a file in php. First, the form, index.php:

<form enctype="multipart/form-data" action="upload.php" method="POST">     <input type="hidden" name="MAX_FILE_SIZE" value="512000" />     Send this file: <input name="userfile" type="file" />     <input type="submit" value="Send File" /> </form> 


Next, the php to accept the file, upload.php

<?php  $uploaddir = '/var/www/uploads/'; $uploadfile = $uploaddir . basename($_FILES['userfile']['name']);  echo "<p>";  if (move_uploaded_file($_FILES['userfile']['tmp_name'], $uploadfile)) {   echo "File is valid, and was successfully uploaded.\n"; } else {    echo "Upload failed"; }  echo "</p>"; echo '<pre>'; echo 'Here is some more debugging info:'; print_r($_FILES); print "</pre>";  ?>  

Getting the _session_id from SWFUpload (Flash 8 multiple file uploader)

It appears that Ruby's CGI::Session class will not use the _session_id in the query string when the Request is a POST.

Normally, a POST-type request occurs when a form is submitted to the server (e.g. a Rails application). In this scenario, there is an easy workaround since we can send the _session_id as a hidden field.

With Flash 8, however, there is no way to add a 'hidden field' to the multi-part form data, thus Rails will fail to recognize the _session_id in the query string portion of our request.

Here is a hackish work-around:

# The following code is a work-around for the # Flash 8 bug that prevents our multiple file uploader # from sending the _session_id.  Here, we hack the # Session#initialize method and force the session_id # to load from the query string via the request uri.  # (Tested on Lighttpd)  class CGI::Session   alias original_initialize initialize   def initialize(request, option = {})     session_key = option['session_key'] || '_session_id'     option['session_id'] =       request.env_table["REQUEST_URI"][0..-1].       scan(/#{session_key}=(.*?)(&.*?)*$/).       flatten.first     original_initialize(request, option)   end end 

XML attributes to database columns

I recently needed to update a database with the contents of an XML file under a controlled environment (details at Blackrat's Blog) and came up with this.

With an XML input file of the following format
<tags>   <tag name='test' desc='Test Description' other='Other Item' />   <tag name='test2' desc='2nd Test Description' other='Another Item' /> </tags> 


and a database which contains identical columns name,desc,other (or a superset of the tags) you can use the follow snippet to populate it.

View [import_xml.rhtml]
<h1>Import XML</h1>  <%= form_tag({ :action => 'import_xml'}, { :multipart => true }) %> <%= file_field 'document', 'file' %> <%= submit_tag 'Import' %> <%= end_form_tag %> 


Controller [names_controller.rb]
def import_xml   require 'rexml/document'   file=params[:document][:file]   doc=REXML::Document.new(file.read)   doc.root.each_element('//tag') do |tag|     @name = Name.new     @name.update_attributes(tag.attributes)   end   redirect_to :action => 'list' end 

Home-brewed file upload in Ruby on Rails

Most of this comes from others' work, but I was able to tool it to my needs and fix some bugs. All of these lines go in the model, which for me has a :file and :content_type attributes. :file stores the complete path to the uploaded file. Be sure to change the string in path_to_file to the place where you want files stored, and that proper permissions are set on that path. Also, sanitize_filename doesn't HAVE to be a private method -- make it public if you want.

### Model ###  def file=(uploaded_file)       @uploaded_file = uploaded_file     @filename = sanitize_filename(@uploaded_file.original_filename)     write_attribute("content_type", @uploaded_file.content_type)   end      def after_create     if !File.exists?(File.dirname(path_to_file))       Dir.mkdir(File.dirname(path_to_file))     end     if @uploaded_file.instance_of?(Tempfile)       FileUtils.copy(@uploaded_file.local_path, path_to_file)     else       File.open(self.path_to_file, "wb") { |f| f.write(@uploaded_file.read) }     end     write_attribute("file", path_to_file)   end    def after_destroy     if File.exists?(self.file)       File.delete(self.file)       Dir.rmdir(File.dirname(self.file))     end   end      def path_to_file     File.expand_path("#{RAILS_ROOT}/upload/#{self.id}/#{@filename}")   end      private   def sanitize_filename(file_name)     # get only the filename, not the whole path (from IE)     just_filename = File.basename(file_name)      # replace all none alphanumeric, underscore or perioids with underscore     just_filename.gsub(/[^\w\.\_]/,'_')    end    ### View ### ... <input type="file" name="model[file]" /> ... 

generic file and image models for uploaded files

These are basic models that store a file in a dedicated files table. Use has_one or has_many to associate this with your actual models. RMagick is required for images.

This is my first code dealing with uploads and rmagick, so please comment if you have suggestions.

class DbFile < ActiveRecord::Base   IMAGE_TYPES = ['image/jpeg', 'image/pjpeg', 'image/gif', 'image/png', 'image/x-png']   before_validation     :sanitize_filename   validates_presence_of :size, :filename, :content_type    class << self     def new_file(file_data)       content_type = file_data.content_type.strip       (IMAGE_TYPES.include?(content_type) ? DbImage : DbFile).new \         :data         => file_data.read,         :filename     => file_data.original_filename,         :size         => file_data.size,         :content_type => content_type     end   end    protected   def sanitize_filename       # NOTE: File.basename doesn't work right with Windows paths on Unix       # get only the filename, not the whole path       filename.gsub! /^.*(\\|\/)/, ''        # Finally, replace all non alphanumeric, underscore or periods with underscore       filename.gsub! /[^\w\.\-]/, '_'   end end  require 'rmagick' require 'base64' class DbImage < DbFile   def data=(file_data)     with_image(file_data, true) do |img|       self.width  = img.columns       self.height = img.rows     end   end    def with_image(file_data = nil, save_image = false, &block)     img = Magick::Image::read_inline(Base64.b64encode(file_data || self.data)).first     block.call(img)     write_attribute('data', img.to_blob) if save_image     img = nil     GC.start   end end


Controller Usage:

# returns DbImage if content_type matches db_file = DbFile.new_file(params[:file][:data]) db_file.save


Model Usage:

# raw binary image data File.open('my_file', 'w') { |f| f.write(db_file.data) }  # Image resizing with rmagick # automatically creates RMagick::Image and  # invokes GC.start db_file.with_image do |img|   img.scale(.25)   img.write('thumb.jpg') end 

Upload file with http client (multipart/form-data)

def post_multipart(host, selector, fields, files):     """     Post fields and files to an http host as multipart/form-data.     fields is a sequence of (name, value) elements for regular form fields.     files is a sequence of (name, filename, value) elements for data to be uploaded as files     Return the server's response page.     """ 

See the rest of the implementation here

Send a file using FTP

import ftplib s = ftplib.FTP('myserver.com','login','password') # Connect  f = open('todo.txt','rb')                # file to send s.storbinary('STOR todo.txt', f)         # Send the file  f.close()                                # Close file and FTP s.quit() 

No comments:

analytics