cjhaas.com

PHP Base85 encode 128-bit integer (GUID/UUID)

Posted in PHP by Chris Haas on November 12th, 2013

I’m working with GUIDs in PHP and am trying to find the most compact way to represent a 128-bit integer via a string. Searches showed theory and C# code along with a Wikipedia article that referenced many languages except for PHP so I decided to roll my own. The code below relies on having the GNU Multiple Precision extension installed which I’ll leave up to you. You can test this by looking at RFC1924 Section 5 which has the number 21932261930451111902915077091070067066 which encodes to 4)+k&C#VzJ4br>0wv%Yp

  /**
   * Converts a 32-character guid string to a 20-character string using Base85
   *
   * @since  0.3.1
   *
   * @see  http://www.mathsisfun.com/binary-decimal-hexadecimal-converter.html  Hex/Dec/Binary conversion with large numbers.
   * @see  http://tools.ietf.org/html/rfc1924                                   Section 5 has sample 128-bit integers in different formats.
   *
   * @param  string $guid_as_hex_string A guid expressed as a hexidecimal string.
   * @return string                     A 20-character representation of the string encoded as base-85.
   */
  public static function convert_guid_to_base85($guid_as_hex_string){
    //Remove any non-guid characters
    $guid_as_hex_string = preg_replace('/[^0-9A-F]/i', '', $guid_as_hex_string);

    //Not a GUID, fail
    if(strlen($guid_as_hex_string) !== 32){
      return false;
    }

    //Possible characters
    $chars = array('0', '1', '2', '3', '4', '5', '6', '7', '8', '9',
                   'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z',
                   'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z',
                   '!', '#', '$', '%', '&', '(', ')', '*', '+', '-', ';', '<', '=', '>', '?', '@', '^', '_', '`', '{', '|', '}', '~'
                  );

    //Build all 20 (nineteen through zero) powers of 85
    $powers = array();
    for($i = 19; $i >= 0; $i--){
      $powers[] = gmp_pow(85, $i);
    }

    //Create our large integer
    $t = gmp_init($guid_as_hex_string, 16);

    //Buf will be our return string, initialize as empty
    $buf = '';

    //Loop through each power in decending order
    foreach($powers as $pow){

      //Divide the current large number by the current power of 85
      //Returns an array, the first element is the integer division of the two, the second is remainder
      $ir = gmp_div_qr($t, $pow);

      //If the result is greater than or equal to 1
      if(gmp_cmp($ir[0], 1) >= 0){
        //Get the character at the integer representation of that position
        $buf .= $chars[gmp_intval($ir[0])];
      }else{
        //Otherwise get the character at the zero position
        $buf .= $chars[0];
      }

      //Reset our large number to just the remainder of the division
      $t = $ir[1];

      //Repeat
    }

    return $buf;
  }

This code isn’t getting run a million times per day so for future reading purposes I’m building the power array manually. The long form of it would be

    $powers = array(
                    gmp_pow(85, 19),
                    gmp_pow(85, 18),
                    gmp_pow(85, 17),
                    gmp_pow(85, 16),
                    gmp_pow(85, 15),
                    gmp_pow(85, 14),
                    gmp_pow(85, 13),
                    gmp_pow(85, 12),
                    gmp_pow(85, 11),
                    gmp_pow(85, 10),
                    gmp_pow(85,  9),
                    gmp_pow(85,  8),
                    gmp_pow(85,  7),
                    gmp_pow(85,  6),
                    gmp_pow(85,  5),
                    gmp_pow(85,  4),
                    gmp_pow(85,  3),
                    gmp_pow(85,  2),
                    gmp_pow(85,  1),
                    gmp_pow(85,  0),
                  );

And if you really want to optimize things (possibly) then you can just use the actual numbers. I didn’t bother testing perf on gmp so I don’t know (or care) what’s faster, math or string parsing.

    $powers = array(
                    gmp_init('4559944833472277161543903350830078125', 10),
                    gmp_init('53646409805556201900516510009765625', 10),
                    gmp_init('631134233006543551770782470703125', 10),
                    gmp_init('7425108623606394726715087890625', 10),
                    gmp_init('87354219101251702667236328125', 10),
                    gmp_init('1027696695308843560791015625', 10),
                    gmp_init('12090549356574630126953125', 10),
                    gmp_init('142241757136172119140625', 10),
                    gmp_init('1673432436896142578125', 10),
                    gmp_init('19687440434072265625', 10),
                    gmp_init('231616946283203125', 10),
                    gmp_init('2724905250390625', 10),
                    gmp_init('32057708828125', 10),
                    gmp_init('377149515625', 10),
                    gmp_init('4437053125', 10),
                    gmp_init('52200625', 10),
                    gmp_init('614125', 10),
                    gmp_init('7225', 10),
                    gmp_init('85', 10),
                    gmp_init('1', 10),
                  );

Add additional keys to WP Remote

Posted in Plugins,WordPress by Chris Haas on November 7th, 2013
/**
 * Add aditional API keys
 */
add_filter('wpr_api_keys', 'vendi__wpr_api_keys');
function vendi__wpr_api_keys($keys){
    if(!is_array($keys)){
        $keys = (array)$keys;    //The stored key is probably a string literal so turn it into an array
    }
    $keys[] = 'SECOND_KEY_HERE'; //Make a note so you know what it was
    $keys[] = 'THIRD_KEY_HERE';  //Make a note so you know what it was
    return $keys;                //Return the above changes
}
Tagged with:

Additional string formatting for $wpdb->insert

Posted in WordPress by Chris Haas on November 5th, 2013

I really like using the helper $wpdb class for inserts but occasionally I have a need to insert data using more than just %s, %d or %f format strings. Almost everyone will tell you to manually write your own INSERT statement and then just pass that to $wpdb->prepare() (which $wpdb->insert() does for you) but there is another way!

And it is actually documented in the code! Kind of. In wp-db on line 1307 (as of WP 3.7.1) it says:

A format is one of ‘%d’, ‘%f’, ‘%s’ (integer, float, string). If omitted, all values in $data will be treated as strings unless otherwise specified in wpdb::$field_types.

The $wpdb has a member variable called $field_types which is an associative that you can use to override the formatting. To use it you cannot specify any formatting parameters when calling insert.

In my case, I had a need insert a GUID-like string into binary(16) column using the UNHEX() MySQL function.

global $wpdb;

//Set out override (anything not specified will be treated as %s)
$wpdb->field_types['transaction_id'] = "UNHEX('%s')";

//Call the normal insert
$wpdb->insert(
             $wpdb->payments,
             array(
                   'transaction_id' => $transaction_id,
                   'name'           => $name
                  )
             );

//Be polite, reset the array just in case future statements are run on this with the same column names
$wpdb->field_types = array();
Tagged with:

CSS Transitions not working first time in Firefox and Internet Explorer

Posted in Uncategorized by Chris Haas on May 24th, 2013

I was doing some simple CSS transitions like transition: left 0.5s ease-in-out 1 and they were working just fine in Chrome but Firefox and Internet Explorer either wouldn’t use the transitions or they wouldn’t work the first time but subsequent times they would. So if I had an arrow that moved something the first click would jump the object while every click after the object would slide.

Simple fix, I wasn’t giving the object an initial value. I thought (incorrectly) that left and top were defaulted to 0 but really they’re null (or whatever CSS’s equivalent of null is).

So I had a declaration like:

#obj{position:absolute;transition: left 0.5s ease-in-out 1;}
#obj:hover{left:-100px;}

But since I never declared left:0 in the first line Firefox and Internet Explorer won’t transition the property. The transition not working the first time but all subsequent times was the same thing, I was setting an initially null property via JavaScript so the first time those browsers had nothing to transition from. Once set they did and everything worked fine.

So moral of the story, if you transition a property, make sure you give it an initial value, too.

UPDATE

Well, the above gets you almost there. You also need to make sure that your timing values also specify their unit. Instead of just 1 or even 0 you need to specify 1s or 0s. Firefox apparently doesn’t have a default unit (even for zero apparently).

#obj{position:absolute;transition: left 0.5s ease-in-out 1s;}
#obj:hover{left:-100px;}
Tagged with:

DO NOT SEND ME MY PASSWORD. EVER!!!

Posted in Uncategorized by Chris Haas on May 17th, 2013

I can’t believe anyone would actually transmit a password in the clear in this day and age. Especially a tech(-ish) company. Just signed up for an API key at a screen shot place (won’t name but people can guess) and they sent me my password via email. Thank you for passing that through untold servers and logging systems!

*** failed to import extension kilnauth from ~/KilnExtensions/kilnauth.py: invalid syntax

Posted in Uncategorized by Chris Haas on April 18th, 2013

You might run into this error message if you’ve got an older version of python installed. The problem is the newer with syntax:

        with open(self.__temporary_path, 'r') as f:
            before = md5(f.read()).digest()

The solution is to just convert it to try/finally

        f = open(self.__temporary_path, 'rb')
        try:
            before = md5(f.read()).digest()
        finally:
            f.close()

MySqlDump to individual files and restoring again

Posted in mysql by Chris Haas on April 17th, 2013

Make sure that your mysql user has permission to access the file system (the FILE privilege). GRANT ALL does not do this. You can check your user by doing:

select user, host, file_priv from mysql.user;

To enable:

grant FILE on *.* to 'root'@'localhost';flush privileges;

Also make sure that mysqld can write to the specified folder (./sql below). To dump:

mysqldump -uroot -p your_database --tab=./sql

Importing back is a two step process, DDL first followed by tab-delimited data:

DDL

cat sql/*.sql | mysql -uroot -p your_database

Tab delimited data

NOTE: The path you provide MUST be absolute. Nevermind, you can use --local to fix
NOTE: The -d parameter means DELETE from the current table before importing!

mysqlimport --local -uroot -p -d your_database sql/*.txt

Additionally, you might want to de-domain your files (for instance, if you’re backing up a WordPress site)

find sql/ -type f -name '*.txt' -exec sed -i 's/old_domain.com/place_holder/g' {} \;

iTextSharp slightly smarter text extraction strategy

Posted in iTextSharp by Chris Haas on March 13th, 2013

iTextSharp’s SimpleTextExtractionStrategy is great but it is simple as the name implies. It can detect new lines pretty well but it has no care for the order of the lines themselves. If your PDF isn’t written top to bottom (as many PDFs aren’t) you’ll get everything out of order. The code below is a modified version of the current SimpleTextExtractionStrategy found here. Instead of just appending text to a master buffer and inserting a newline every time a different Y coordinate is found it stores the Y coordinates in a dictionary and appends to each. There’s actually several flaws in this logic but so far it has been working pretty well for me, at least in comparison to the old way.

I’ve removed many of the comments from the source, see the original link above for details on various fields and methods. Any comments below are my alterations to the above.

using iTextSharp.text.pdf.parser;
using System;
using System.Collections.Generic;
using System.Text;
/*
 * $Id: SimpleTextExtractionStrategy.cs 318 2012-02-27 22:46:07Z psoares33 $
 *
 * This file is part of the iText project.
 * Copyright (c) 1998-2012 1T3XT BVBA
 * Authors: Kevin Day, Bruno Lowagie, Paulo Soares, et al.
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU Affero General Public License version 3
 * as published by the Free Software Foundation with the addition of the
 * following permission added to Section 15 as permitted in Section 7(a):
 * FOR ANY PART OF THE COVERED WORK IN WHICH THE COPYRIGHT IS OWNED BY 1T3XT,
 * 1T3XT DISCLAIMS THE WARRANTY OF NON INFRINGEMENT OF THIRD PARTY RIGHTS.
 *
 * This program is distributed in the hope that it will be useful, but
 * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
 * or FITNESS FOR A PARTICULAR PURPOSE.
 * See the GNU Affero General Public License for more details.
 * You should have received a copy of the GNU Affero General Public License
 * along with this program; if not, see http://www.gnu.org/licenses or write to
 * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
 * Boston, MA, 02110-1301 USA, or download the license from the following URL:
 * http://itextpdf.com/terms-of-use/
 *
 * The interactive user interfaces in modified source and object code versions
 * of this program must display Appropriate Legal Notices, as required under
 * Section 5 of the GNU Affero General Public License.
 *
 * In accordance with Section 7(b) of the GNU Affero General Public License,
 * you must retain the producer line in every PDF that is created or manipulated
 * using iText.
 *
 * You can be released from the requirements of the license by purchasing
 * a commercial license. Buying such a license is mandatory as soon as you
 * develop commercial activities involving the iText software without
 * disclosing the source code of your own applications.
 * These activities include: offering paid services to customers as an ASP,
 * serving PDFs on the fly in a web application, shipping iText with a closed
 * source product.
 *
 * For more information, please contact iText Software Corp. at this
 * address: [email protected]
 */
public class TopToBottomTextExtractionStrategy : ITextExtractionStrategy {

    private Vector lastStart;
    private Vector lastEnd;

    //Store each line individually. A SortedDictionary will automatically shuffle things around based on the key
    private SortedDictionary<int, StringBuilder> results = new SortedDictionary<int, StringBuilder>();

    //Constructor and some methods that aren't used
    public TopToBottomTextExtractionStrategy() { }
    public virtual void BeginTextBlock() { }
    public virtual void EndTextBlock() { }
    public virtual void RenderImage(ImageRenderInfo renderInfo) { }

    //Convert our lines into a giant block of text
    public virtual String GetResultantText() {
        //Buffer
        StringBuilder buf = new StringBuilder();
        //Loop through each line (which is already sorted top to bottom)
        foreach (var s in results) {
            //Append to the buffer
            buf.AppendLine(s.Value.ToString());
        }
        return buf.ToString();
    }
    public virtual void RenderText(TextRenderInfo renderInfo) {
        bool firstRender = results.Count == 0;

        LineSegment segment = renderInfo.GetBaseline();
        Vector start = segment.GetStartPoint();
        Vector end = segment.GetEndPoint();

        //Use the Y value of the bottom left corner of the text for the key
        int currentLineKey = (int)start[1];

        if (!firstRender) {
            Vector x0 = start;
            Vector x1 = lastStart;
            Vector x2 = lastEnd;

            float dist = (x2.Subtract(x1)).Cross((x1.Subtract(x0))).LengthSquared / x2.Subtract(x1).LengthSquared;

            float sameLineThreshold = 1f;
            //If we've detected that we're still on the same
            if (dist <= sameLineThreshold) {
                //Use the previous Y coordinate
                currentLineKey = (int)lastStart[1];
            }
        }
        //Hack: PDFs start with zero at the bottom so our keys will be upside down. Using negative keys cheats this.
        currentLineKey = currentLineKey * -1;

        //If this line hasn't been used before add a new line to our collection
        if (!results.ContainsKey(currentLineKey)) {
            results.Add(currentLineKey, new StringBuilder());
        }

        //Insert a space between blocks of text if it appears there should be
        if (!firstRender &&                                       //First pass never needs a leading space
            results[currentLineKey].Length !=0 &&                 //Don't append a space to the begining of a line
            !results[currentLineKey].ToString().EndsWith(" ") &&  //Don't append if the current buffer ends in a space already
            renderInfo.GetText().Length > 0 &&                    //Don't append if the new next is empty
            !renderInfo.GetText().StartsWith(" ")) {              //Don't append if the new text starts with a space
            //Calculate the distance between the two blocks
            float spacing = lastEnd.Subtract(start).Length;
            //If it "looks" like it should be a space
            if (spacing > renderInfo.GetSingleSpaceWidth() / 2f) {
                //Add a space
                results[currentLineKey].Append(" ");
            }
        }

        //Add the text to the line in our collection
        results[currentLineKey].Append(renderInfo.GetText());

        lastStart = start;
        lastEnd = end;
    }
}

WordPress Post Formats Admin UI

Posted in Uncategorized by Chris Haas on January 24th, 2013

Error when resizing VirtualBox disk on Windows

Posted in Uncategorized by Chris Haas on November 27th, 2012

I recently needed to resize my CentOS VDI using the following command:

"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" modifyhd "c:\CentOS\CentOS.vdi" --resize 20480

Unfortunately it kept failing with this error message:

VBoxManage.exe: error: Failed to create the VirtualBox object!
VBoxManage.exe: error: Code CO_E_SERVER_EXEC_FAILURE (0x80080005) - Server execution failed (extended info not available)
VBoxManage.exe: error: Most likely, the VirtualBox COM server is not running or failed to start.

After pulling my hair for almost an hour I finally tried running the command from a NON-elevated command prompt. Yes, that’s right, this command apparently MUST NOT be run with administrator privileges. Weird.

Also, this is CentOS with an LVM partition which can’t be resized with GParted. To resize an LVM partition follow these steps.