A geek stranded on Martha’s Vineyard
How to Build GnuPG 2 from Source on OS X
I’m a big fan of homebrew, but when it comes to encryption, I get paranoid and prefer to build the software myself.
On the other hand, building Unix software from source is actually pretty simple. If you’ve never done it this is a good opportunity to try. GnuPG requires several packages so you’ll be able to exercise several times.
Versions
These instructions were tested with gnupg-2.2.9 on macOS High Sierra 10.13.6, but they should be useful for other versions as well.
List of Packages
GnuPG requires installing several packages, the download page has all of them, but this is more than you need. It’s better to follow the list that comes in the README instead, so let’s start downloading and extracting GnuPG.
The location of the tar file comes from the download link on the download page
cd /usr/local/src # prefered location for sources curl -o gnupg-2.2.9.tar.bz2 \ https://gnupg.org/ftp/gcrypt/gnupg/gnupg-2.2.9.tar.bz2
Before uncompressing the tarball you need to verify that this is the original and umodified version of the source. If you have an existing installation of gpg you can verify its signature. Otherwise you have to compare the file checksum, which is what I did. The list of checksums is on the integrity page.
openssl sha1 gnupg-2.2.9.tar.bz2
Once you have verified that the checksum of the output of this command matches the checksum listed on integrity page you can uncompress it.
tar -xvzf gnupg-2.2.9.tar.bz2 cd gnupg-2.2.9 cat README
The BUILD INSTRUCTIONS section of the README lists the required packages. It also has all the instructions you need.
npth | https://gnupg.org/ftp/gcrypt/npth/ |
libgpg-error | https://gnupg.org/ftp/gcrypt/libgpg-error/ |
libgcrypt | https://gnupg.org/ftp/gcrypt/libgcrypt/ |
libksba | https://gnupg.org/ftp/gcrypt/libksba/ |
libassuan | https://gnupg.org/ftp/gcrypt/libassuan/ |
pinentry | https://gnupg.org/ftp/gcrypt/pinentry/ |
dirmngr | (if you’re going to use network commands) |
Building from Source
The steps to build from source are the same for every package, for example, for npth they would be:
cd /usr/local/src curl -o npth-1.6.tar.bz2 \ https://gnupg.org/ftp/gcrypt/npth/npth-1.6.tar.bz2 openssl sha1 npth-1.6.tar.bz2 # compare the output against gnupg.org/download/integrity_check.html tar -xvzf npth-1.6.tar.bz2 cd npth-1.6 ./configure # no options except for pinentry (see below) make make check make install
The ./configure
script gathers information about your environment: the command to invoke the C compiler, location of libraries, etc. It verifies that all dependencies are met. It also generates the Makefile and defines some constants.
Calling make
will compile all sources using the Makefile generated by the previous step.
The make check
is not required but a good idea to detect any issues. In the case of libgcrypt it also runs some benchmarks, but for me they run for too long and I ended up canceling it.
Sometimes make
or make check
will issue errors. I didn’t run into any in this case, but in the past, for older versions of GPG, I did get one, when the location of files was different; in other instance some constant definitions were missing.
make install
will copy executables and libraries and create symbolic links in directories where they can be accessed. Sometimes you might need extra permissions to finish this step and have to execute using sudo
. It didn’t happen to me.
Pinentry Options
For all the packages I used the defaults, i.e. didn’t provide any options to ./configure
This was the exception. Pinentry is used to enter passwords/passphrases, since I’m a terminal-type person I like to use tty:
./configure \ --disable-pinentry-qt \ --disable-pinentry-emacs \ --disable-inside-emacs \ --disable-pinentry-gtk2 \ --disable-pinentry-curses \ --enable-pinentry-tty
When you run this command the output should be:
Pinentry v1.1.0 has been configured as follows: [output omitted] Default Pinentry .: pinentry-tty
Finally Build GnuPG
Once all the packages are done you should be ready to finish building GnuPG:
cd /usr/local/src/gnupg-2.2.9 ./configure make make check make install
dirmgr
When I tried to use --recv-keys
or --send-keys
I got this error:
gpg: failed to start the dirmngr '/usr/local/bin/dirmngr': No such file or directory
dirmgr is used by GnuPG to perform network operations. It’s actually part of the gpg package and has already been compiled and linked when you built it. The only step missing is installation, you do need to use sudo
in this case:
cd /usr/local/src/gnupg-2.2.9/dirmngr sudo make install
Presenting at RailsConf 2017
One of the benefits of working at ActBlue is that you have the opportunity to play with lots of traffic.
In August of last year, I gave a talk at Boston Ruby Group sharing our experience building a high performance web application. The feedback I received from that audience and from colleagues allowed me to improve it and submit a proposal to RailsConf 2017.
My application was accepted in the High Volume track. I gave my talk last April, and although I was pretty nervous, I’m happy with the result. After 6 months, the talk is ranked #26 of 86 with 500 views in Confreaks. I recommend you to watch it.
Understanding Unicode Encoding in Ruby by Example
Every time I have to troubleshoot a problem with Unicode, it takes time to go through the documentation.
I compiled this list of methods and examples of how to use them. It has proven to save me time by quickly refreshing my memory.
Tested with Ruby 2.4.1 on macOS Sierra 10.12.4.
# the encoding is a property of String utf8_resume = "Résumé" => "Résumé" utf8_resume.encoding => #<Encoding:UTF-8> # translate the same string to different encodings latin1_resume = utf8_resume.encode("ISO-8859-1") latin9_resume = utf8_resume.encode("ISO-8859-15") utf8_resume.encoding => #<Encoding:UTF-8> latin1_resume.encoding => #<Encoding:ISO-8859-1> latin9_resume.encoding => #<Encoding:ISO-8859-15> # specify the string using codepoints lower_spanish_accents = "\u00E1\u00E9\u00ED\u00F3\u00FA\u00F1".encode("UTF-8") => "áéíóúñ" upper_spanish_accents = "\u00C1\u00C9\u00CD\u00D3\u00DA\u00D1".encode("UTF-8") => "ÁÉÍÓÚÑ" # Length of the encoded text # in UTF-8 # 'z' is 1 byte # 'ñ' is 2 bytes z = "\u007A" => "z" z.each_byte.map{|c| "%X" % c} => ["7A"] n_tilde = "\u00F1" => "ñ" n_tilde.each_byte.map{|c| "%X" % c} => ["C3", "B1"] n_tilde.bytesize => 2 # but in Latin-1 'ñ' is only 1 byte n_tilde.encode('iso-8859-1').each_byte.map{|c| "%X" % c} => ["F1"] # but Unicode is universal, so in codepoints there's no difference # between UTF-8 and Latin-1 n_tilde.each_codepoint.map {|c| "%X" % c} => ["F1"] n_tilde.codepoints.size => 1 n_tilde.encode('iso-8859-1').each_codepoint.map{|c| "%X" % c} => ["F1"] n_tilde.encode('iso-8859-1').codepoints.size => 1 # codepoints are base-10 integers n_tilde.encode('iso-8859-1').codepoints => [241] n_tilde.codepoints => [241] # formats to specify codepoints # single codepoint # exactly 4 hex digits # \uXXXX <==> U+XXXX # multiple codepoints # hex digits # leading 0 is optional # \u{X XX XXX XXXX} <==> U+000X U+00XX U+0XXX U+XXXX "\u0045\u0073\u0070\u0061\u00F1\u0061" => "España" "\u{45 73 70 61 F1 61}" => "España" # sometimes codepoint and byte sequence will match "\u007f".each_codepoint.map{|c| "%X" % c} => ["7F"] "\u007f".each_byte.map{|c| "%X" % c} => ["7F"] # but this isn't always true # see also example for ñ above "\u0080".each_codepoint.map{|c| "%X" % c} => ["80"] "\u0080".each_byte.map{|c| "%X" % c} => ["C2", "80"] # not all byte sequences are valid encodings "\u3042".valid_encoding? => true "\u3042\x81".valid_encoding? => false # scrub to the rescue scrubbed = "\u3042\x81".scrub('') scrubbed.valid_encoding? => true scrubbed.each_codepoint.map{|c| "%X" % c} => ["3042"] # building the string using the internal representation, i.e. byte by byte espana_utf8 = [0x45, 0x73, 0x70, 0x61, 0xc3, 0xb1, 0x61] => [69, 115, 112, 97, 195, 177, 97] espana_utf8.pack('c*').force_encoding('utf-8') => "España" # now in Latin1 (different byte sequence) espana_latin1 = [0x45, 0x73, 0x70, 0x61, 0xf1, 0x61] => [69, 115, 112, 97, 241, 97] espana_latin1.pack('c*').force_encoding('ISO-8859-1') => "Espa\xF1a" # although the ñ doesn't look correct, the enconding is correct espana_latin1.pack('c*').force_encoding('ISO-8859-1').valid_encoding? => true # currency symbols in UTF-8 currency_utf8 = "\u{20AC A3 A5}" => "€£¥" # Convert a number from any base to any base class String def convert_base(from, to) to_i(from).to_s(to) end end # example: letter "~", from base 16 to base 10 "7E".convert_base(16, 10) => "126" # example: decimal 255 to hexadecimal '255'.convert_base(10, 16) => "ff"
How to Upgrade to Strong Parameters in Rails
This article was originally published in the blog of ActBlue Technical Services.
In the previous blog post, using a series of tests, we described how strong parameters work. In this post we detail the steps we followed to upgrade our main application in ActBlue.
Starting Point
To help with the process Rails provides the protected_attributes
gem. This allows you to run your app on 4.1 without having to make any changes related to mass-assignment protection. The gem brings backwards compatibility by implementing attr_accessible
, attr_protected
and other methods.
Add these lines to your Gemfile, run bundle install
and make all your tests pass (you have tests, right?)
gem 'rails', '~> 4.1' gem 'protected_attributes'
There are other incompatibilities you will have to resolve, but this post is about strong parameters and we are assuming your tests run green at this point.
Mixing Both
It is unlikely you can make all the changes in a single release and therefore you will want to have both protected attributes and strong parameters working at the same time. So the next step is to add this line to Gemfile and run bundle install
:
gem 'strong_parameters'
In every model you want to upgrade add this line to include the ForbiddenAttributesProtection
module. This is the way to indicate Rails which models are using the new mechanism, for example:
class Book < ActiveRecord::Base include ActiveModel::ForbiddenAttributesProtection end
In the model also remove all calls to attr_accessible
and attr_protected
, for example:
attr_accessible :isbn, :title attr_protected :price
In the corresponding controller it is useful to create a private method that whitelists its parameters, for instance:
private def book_params params.require(:book).permit(:isbn, :title) end
For simple models and controllers it is going to be similar to this, but for more complex cases you can check our previous blog post or the documentation.
Dropping Protected Attributes
When you think you have upgraded all your models and controllers and feel you are ready to pull the plug on the old protected attributes, these are the steps we recommend:
Remove these 2 lines from Gemfile and run bundle install
.
gem 'protected_attributes' gem 'strong_parameters'
In Rails 4 the default is strong parameters, so there is no need to include the gem.
Remove the configuration for protected_attributes, this means removing from config/application.rb:
config.active_record.whitelist_attributes = false
Remove from config/environments/test.rb:
config.active_record.mass_assignment_sanitizer = :logger
Remove any include ActiveModel::ForbiddenAttributesProtection
from models, added in the previous step.
In the default configuration of strong parameters, when you whitelist only a subset of the params passed to the controller, instead of raising an exception, Rails is going to generate a notification. You can see a test illustrating this.
It is better instead to always raise an exception. This is done by adding this line to config/environments/test.rb:
config.action_controller.action_on_unpermitted_parameters = :raise
Now run all your tests. If you have a large application and you are getting errors you do not understand or the code is not behaving the way you expect, check our previous blog post Understanding Strong Parameters, this is the part where this information is most useful.
Final Step
When you are done with the tests you can remove the line from config/environments/test.rb:
config.action_controller.action_on_unpermitted_parameters = :raise
The default value in Rails is to generate a notification in test and development environments, and ignore otherwise.
You are done! We hope the 2 articles helped you to make the transition smoother.