upload_to_s3 - Ruby S3 upload client
Prerequisites:
gem install aws-s3
gem install main
gem install aws-s3
gem install main
#!/bin/env ruby require 'rubygems' require 'main' require 'aws/s3' include AWS::S3 Main { argument('source_filename') { cast :string description 'source filename to copy to S3' } argument('bucket_name') { cast :string description 'bucket to place the file in on S3' } option('access_key_id') { argument :optional description 'specify the access_key_id manually' default 'put your access key here if you want' } option('secret_access_key') { argument :optional description 'specify the secret key manually' default 'put your secret key here if you want' } def run bucket_name = params['bucket_name'].value source_filename = params['source_filename'].value Base.establish_connection!( :access_key_id => params['access_key_id'].value, :secret_access_key => params['secret_access_key'].value ) begin Bucket.find(bucket_name) rescue puts "Need to make bucket #{bucket_name}.." Bucket.create(bucket_name) # Confirm its existence.. Bucket.find(bucket_name) end puts "Got bucket.." puts "Uploading #{File.basename(source_filename)}.." S3Object.store(File.basename(source_filename), open(source_filename), bucket_name) puts "Stored!" exit_success! end }
Make a remote URL work like a file upload (in Rails)
Want to load a remote URL into an acts_as_attachment/attachment_fu model? Use this little utility class.
Now when you have the URL you want to load, do something like this:
Or better yet, make a pseudo-accessor on your aaa/attachment_fu model so you can stay "model-heavy".
class UrlUpload EXTENSIONS = { "image/jpeg" => ["jpg", "jpeg", "jpe"], "image/gif" => ["gif"], "image/png" => ["png"] } attr_reader :original_filename, :attachment_data def initialize(url) @attachment_data = open(url) @original_filename = determine_filename end # Pass things like size, content_type, path on to the downloaded file def method_missing(symbol, *args) if self.attachment_data.respond_to? symbol self.attachment_data.send symbol, *args else super end end private def determine_filename # Grab the path - even though it could be a script and not an actual file path = self.attachment_data.base_uri.path # Get the filename from the path, make it lowercase to handle those # crazy Win32 servers with all-caps extensions filename = File.basename(path).downcase # If the file extension doesn't match the content type, add it to the end, changing any existing .'s to _ filename = [filename.gsub(/\./, "_"), EXTENSIONS[self.content_type].first].join(".") unless EXTENSIONS[self.content_type].any? {|ext| filename.ends_with?("." + ext) } # Return the result filename end end
Now when you have the URL you want to load, do something like this:
@model.uploaded_data = UrlUpload.new(url)
Or better yet, make a pseudo-accessor on your aaa/attachment_fu model so you can stay "model-heavy".
def url=(value) self.uploaded_data = UrlUpload.new(value) end
Very simple php file upload
I think this is the minimum necessary to upload a file in php. First, the form, index.php:
Next, the php to accept the file, upload.php
<form enctype="multipart/form-data" action="upload.php" method="POST"> <input type="hidden" name="MAX_FILE_SIZE" value="512000" /> Send this file: <input name="userfile" type="file" /> <input type="submit" value="Send File" /> </form>
Next, the php to accept the file, upload.php
<?php $uploaddir = '/var/www/uploads/'; $uploadfile = $uploaddir . basename($_FILES['userfile']['name']); echo "<p>"; if (move_uploaded_file($_FILES['userfile']['tmp_name'], $uploadfile)) { echo "File is valid, and was successfully uploaded.\n"; } else { echo "Upload failed"; } echo "</p>"; echo '<pre>'; echo 'Here is some more debugging info:'; print_r($_FILES); print "</pre>"; ?>
Getting the _session_id from SWFUpload (Flash 8 multiple file uploader)
It appears that Ruby's CGI::Session class will not use the _session_id in the query string when the Request is a POST.
Normally, a POST-type request occurs when a form is submitted to the server (e.g. a Rails application). In this scenario, there is an easy workaround since we can send the _session_id as a hidden field.
With Flash 8, however, there is no way to add a 'hidden field' to the multi-part form data, thus Rails will fail to recognize the _session_id in the query string portion of our request.
Here is a hackish work-around:
Normally, a POST-type request occurs when a form is submitted to the server (e.g. a Rails application). In this scenario, there is an easy workaround since we can send the _session_id as a hidden field.
With Flash 8, however, there is no way to add a 'hidden field' to the multi-part form data, thus Rails will fail to recognize the _session_id in the query string portion of our request.
Here is a hackish work-around:
# The following code is a work-around for the # Flash 8 bug that prevents our multiple file uploader # from sending the _session_id. Here, we hack the # Session#initialize method and force the session_id # to load from the query string via the request uri. # (Tested on Lighttpd) class CGI::Session alias original_initialize initialize def initialize(request, option = {}) session_key = option['session_key'] || '_session_id' option['session_id'] = request.env_table["REQUEST_URI"][0..-1]. scan(/#{session_key}=(.*?)(&.*?)*$/). flatten.first original_initialize(request, option) end end
XML attributes to database columns
I recently needed to update a database with the contents of an XML file under a controlled environment (details at Blackrat's Blog) and came up with this.
With an XML input file of the following format
and a database which contains identical columns name,desc,other (or a superset of the tags) you can use the follow snippet to populate it.
View [import_xml.rhtml]
Controller [names_controller.rb]
With an XML input file of the following format
<tags> <tag name='test' desc='Test Description' other='Other Item' /> <tag name='test2' desc='2nd Test Description' other='Another Item' /> </tags>
and a database which contains identical columns name,desc,other (or a superset of the tags) you can use the follow snippet to populate it.
View [import_xml.rhtml]
<h1>Import XML</h1> <%= form_tag({ :action => 'import_xml'}, { :multipart => true }) %> <%= file_field 'document', 'file' %> <%= submit_tag 'Import' %> <%= end_form_tag %>
Controller [names_controller.rb]
def import_xml require 'rexml/document' file=params[:document][:file] doc=REXML::Document.new(file.read) doc.root.each_element('//tag') do |tag| @name = Name.new @name.update_attributes(tag.attributes) end redirect_to :action => 'list' end
Home-brewed file upload in Ruby on Rails
Most of this comes from others' work, but I was able to tool it to my needs and fix some bugs. All of these lines go in the model, which for me has a :file and :content_type attributes. :file stores the complete path to the uploaded file. Be sure to change the string in path_to_file to the place where you want files stored, and that proper permissions are set on that path. Also, sanitize_filename doesn't HAVE to be a private method -- make it public if you want.
### Model ### def file=(uploaded_file) @uploaded_file = uploaded_file @filename = sanitize_filename(@uploaded_file.original_filename) write_attribute("content_type", @uploaded_file.content_type) end def after_create if !File.exists?(File.dirname(path_to_file)) Dir.mkdir(File.dirname(path_to_file)) end if @uploaded_file.instance_of?(Tempfile) FileUtils.copy(@uploaded_file.local_path, path_to_file) else File.open(self.path_to_file, "wb") { |f| f.write(@uploaded_file.read) } end write_attribute("file", path_to_file) end def after_destroy if File.exists?(self.file) File.delete(self.file) Dir.rmdir(File.dirname(self.file)) end end def path_to_file File.expand_path("#{RAILS_ROOT}/upload/#{self.id}/#{@filename}") end private def sanitize_filename(file_name) # get only the filename, not the whole path (from IE) just_filename = File.basename(file_name) # replace all none alphanumeric, underscore or perioids with underscore just_filename.gsub(/[^\w\.\_]/,'_') end ### View ### ... <input type="file" name="model[file]" /> ...
generic file and image models for uploaded files
These are basic models that store a file in a dedicated files table. Use has_one or has_many to associate this with your actual models. RMagick is required for images.
This is my first code dealing with uploads and rmagick, so please comment if you have suggestions.
Controller Usage:
Model Usage:
This is my first code dealing with uploads and rmagick, so please comment if you have suggestions.
class DbFile < ActiveRecord::Base IMAGE_TYPES = ['image/jpeg', 'image/pjpeg', 'image/gif', 'image/png', 'image/x-png'] before_validation :sanitize_filename validates_presence_of :size, :filename, :content_type class << self def new_file(file_data) content_type = file_data.content_type.strip (IMAGE_TYPES.include?(content_type) ? DbImage : DbFile).new \ :data => file_data.read, :filename => file_data.original_filename, :size => file_data.size, :content_type => content_type end end protected def sanitize_filename # NOTE: File.basename doesn't work right with Windows paths on Unix # get only the filename, not the whole path filename.gsub! /^.*(\\|\/)/, '' # Finally, replace all non alphanumeric, underscore or periods with underscore filename.gsub! /[^\w\.\-]/, '_' end end require 'rmagick' require 'base64' class DbImage < DbFile def data=(file_data) with_image(file_data, true) do |img| self.width = img.columns self.height = img.rows end end def with_image(file_data = nil, save_image = false, &block) img = Magick::Image::read_inline(Base64.b64encode(file_data || self.data)).first block.call(img) write_attribute('data', img.to_blob) if save_image img = nil GC.start end end
Controller Usage:
# returns DbImage if content_type matches db_file = DbFile.new_file(params[:file][:data]) db_file.save
Model Usage:
# raw binary image data File.open('my_file', 'w') { |f| f.write(db_file.data) } # Image resizing with rmagick # automatically creates RMagick::Image and # invokes GC.start db_file.with_image do |img| img.scale(.25) img.write('thumb.jpg') end
Upload file with http client (multipart/form-data)
def post_multipart(host, selector, fields, files): """ Post fields and files to an http host as multipart/form-data. fields is a sequence of (name, value) elements for regular form fields. files is a sequence of (name, filename, value) elements for data to be uploaded as files Return the server's response page. """
See the rest of the implementation here
Send a file using FTP
import ftplib s = ftplib.FTP('myserver.com','login','password') # Connect f = open('todo.txt','rb') # file to send s.storbinary('STOR todo.txt', f) # Send the file f.close() # Close file and FTP s.quit()
No comments:
Post a Comment