tag:blogger.com,1999:blog-12183358136805380982024-03-05T01:52:08.787-08:00Home Page ofHelio Perroni Filho"The future is already here...
It's just not evenly distributed." (William Gibson)Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comBlogger74125tag:blogger.com,1999:blog-1218335813680538098.post-46332469654782330562015-12-21T22:51:00.000-08:002015-12-21T22:51:29.379-08:00FFT-based cosine similarity for fast template matching<div dir="ltr" style="text-align: left;" trbidi="on">A common task in computer vision applications is <a href="https://en.wikipedia.org/wiki/Template_matching">template matching</a>, the search for a comparatively small image patch – or more likely, its closest approximation – in a larger image. In its simplest form, this implies solving the following optimization:<br />
</div>\begin{equation}<br />
(i^*, j^*) = \arg \max_{i,j} s(T, P_{i,j})<br />
\label{eq:tm}<br />
\end{equation}<div dir="ltr" style="text-align: left;" trbidi="on">Where the similarity metric \(s(T, P_{i,j})\) is defined between a template \(T^{m_t \times n_t}\) and all patches \(P^{m_t \times n_t}_{i,j}\) such that:<br />
</div>\begin{equation}<br />
P^{m_t \times n_t}_{i,j} = I[i:i+m_t,j:j+n_t]<br />
\label{eq:p}<br />
\end{equation}<div dir="ltr" style="text-align: left;" trbidi="on">For an image \(I^{m \times n}\) with \(m_t \ll m\) and \(n_t \ll n\) (i.e. the image is much larger than the template). A common choice of similarity metric is <a href="https://en.wikipedia.org/wiki/Cosine_similarity">cosine similarity</a>, which is defined as:<br />
</div>\begin{equation}<br />
cos(A, B) = \frac{A \cdot B}{\|A\| \|B\|} = \frac{\displaystyle\sum^{m,n}_{i,j} A[i,j]B[i,j]}{\sqrt{\displaystyle\sum^{m,n}_{i,j} A[i,j]^2 \displaystyle\sum^{m,n}_{i,j} B[i,j]^2}}<br />
\label{eq:cos}<br />
\end{equation}<br />
<div dir="ltr" style="text-align: left;" trbidi="on">Cosine similarity is a reliable metric, but as defined above its evaluation is rather costly. Combining formulas \eqref{eq:tm}, \eqref{eq:p} and \eqref{eq:cos} gives:<br />
</div>\begin{equation}<br />
(i^*, j^*) = \arg \max_{i,j} \frac{\displaystyle\sum^{m_t,n_t}_{i_t,j_t} T[i_t,j_t] I[i + i_t, j + j_t]}{\sqrt{\displaystyle\sum^{m_t,n_t}_{i_t,j_t} T[i_t,j_t]^2 \displaystyle\sum^{m_t,n_t}_{i_t,j_t} I[i + i_t, j + j_t]^2}}<br />
\label{eq:tm_cos}<br />
\end{equation}<div dir="ltr" style="text-align: left;" trbidi="on">Which is of time complexity \(\mathrm{O}(m \, n \, m_tn_t)\) – i.e. the size of the template times the size of the image. Depending on image and template sizes, the intended use case (e.g. batch vs. real-time) or scale of the template matching task (e.g. a couple dozen vs. thousands of images), this may incur in prohibitive processing costs.<br />
</div><br />
<div dir="ltr" style="text-align: left;" trbidi="on">Cosine similarity template matching can be sped up with application of the <a href="https://en.wikipedia.org/wiki/Convolution_theorem">convolution theorem</a>. First, let's redefine formula \eqref{eq:tm_cos} as:<br />
</div>\begin{equation}<br />
(i^*, j^*) = \arg \max_{i,j} \frac{D_{T, I}[i,j]}{\sqrt{m_T M_I[i,j]}}<br />
\label{eq:tm_cos_2}<br />
\end{equation}<div dir="ltr" style="text-align: left;" trbidi="on">Where:<br />
</div>\begin{equation}<br />
D_{T, I}[i,j] = \displaystyle\sum^{m_t,n_t}_{i_t,j_t} T[i_t,j_t] I[i + i_t, j + j_t]<br />
\label{eq:D_TI}<br />
\end{equation}<br />
\begin{equation}<br />
m_T = \displaystyle\sum^{m_t,n_t}_{i_t,j_t} T[i_t,j_t]^2<br />
\label{eq:m_T}<br />
\end{equation}<br />
\begin{equation}<br />
M_I[i,j] = \displaystyle\sum^{m_t,n_t}_{i_t,j_t} I[i + i_t, j + j_t]^2<br />
\label{eq:M_I}<br />
\end{equation}<div dir="ltr" style="text-align: left;" trbidi="on">Looking into the three terms above, it's clear that \eqref{eq:m_T} is constant for any given \(T\) and can therefore be left out of the computation. On the other hand, \eqref{eq:D_TI} is just the <a href="https://en.wikipedia.org/wiki/Cross-correlation">cross-correlation</a> between \(T\) and \(I\), which by the convolution theorem can also be computed as:<br />
</div>\begin{equation}<br />
D_{T, I} = \mathcal{F}^{-1}(\mathcal{F}(T)^{*} \circ \mathcal{F}(I))<br />
\end{equation}<div dir="ltr" style="text-align: left;" trbidi="on">Where \(\mathcal{F}\) is the <a href="https://en.wikipedia.org/wiki/Fourier_transform">fourier transform</a> operator, \(\mathcal{F}^{-1}\) the inverse transform, the asterisk denotes the <a href="https://en.wikipedia.org/wiki/Complex_conjugate">complex conjugate</a>, and \(\circ\) is the <a href="https://en.wikipedia.org/wiki/Hadamard_product_%28matrices%29">Hadamard product</a> (element-wise multiplication) of the two transforms. Likewise, \eqref{eq:M_I} can be computed as the cross-correlation between \(I^2\) and a window filter \(W^{m_t \times n_t} = [w_{ij} = 1]\):<br />
</div>\begin{equation}<br />
M_I = \mathcal{F}^{-1}(\mathcal{F}(W)^{*} \circ \mathcal{F}(I^2))<br />
\end{equation}<div dir="ltr" style="text-align: left;" trbidi="on">The advantage of this approach is that algorithms such as the <a href="https://en.wikipedia.org/wiki/Fast_Fourier_transform">Fast Fourier Transform</a> (FFT) are of time complexity \(\mathrm{O}(m \, n \, log \, m \, n)\), which depending on the relative sizes of \(T\) and \(I\) may be faster than the \(\mathrm{O}(m \, n \, m_t n_t)\) of the direct computation. Also the Fourier transforms of \(W\) and (if the same template will be applied to several images) \(T\) can be cached, further saving up computation time.<br />
</div><br />
<div dir="ltr" style="text-align: left;" trbidi="on"><b>Implementation</b><br />
</div><br />
<div dir="ltr" style="text-align: left;" trbidi="on">The C++ code below provides a basic implementation of the method outlined above. It uses the popular <a href="http://opencv.org/">OpenCV</a> library, with some further optimizations particular to its implementation of the Fourier transform, explained in the comments.<br />
</div><pre class="prettyprint">#include <opencv2/opencv.hpp>
static const cv::Scalar ONE(1);
static const cv::Scalar ZERO(0);
static const cv::Scalar WHITE(255, 255, 255);
// Fourier transform performance is not a monotonic function of a vector
// size - matrices whose dimensions are powers of two are the fastest to
// process, and multiples of 2, 3 and 5 (for example, 300 = 5*5*3*2*2) are
// also processed quite efficiently. Therefore it makes sense to pad input
// data with zeros to get a bit larger matrix that can be transformed much
// faster than the original one.
cv::Size fit(const cv::Size &size)
{
return cv::Size(cv::getOptimalDFTSize(size.width),
cv::getOptimalDFTSize(size.height));
}
cv::Mat F_fwd(const cv::Mat &I, const cv::Size &size)
{
// Pad input matrix to given size.
cv::Mat P;
int m = size.height - I.rows;
int n = size.width - I.cols;
cv::copyMakeBorder(I, P, 0, m, 0, n, cv::BORDER_CONSTANT, ZERO);
// Compute Fourier transform for input data. The last argument
// informs the dft() function of how many non-zero rows are there,
// so it can handle the rest of the rows more efficiently and save
// some time.
cv::Mat F;
cv::dft(P, F, 0, I.rows);
return F;
}
cv::Mat F_inv(const cv::Mat &F, const cv::Size &size)
{
// Compute inverse Fourier transform for input data. The last
// argument informs the dft() function of how many non-zero
// rows are expected in the output, so it can handle the rest
// of the rows more efficiently and save some time.
cv::Mat I;
cv::dft(F, I, cv::DFT_INVERSE + cv::DFT_SCALE, size.height);
return I(cv::Rect(0, 0, size.width, size.height));
}
cv::Mat C(const cv::Mat &T, const cv::Mat &I, const cv::Size &size)
{
// Compute the Fourier transforms of template and image.
cv::Mat F_T = F_fwd(T, size);
cv::Mat F_I = F_fwd(I, size);
// Compute the cross correlation in the frequency domain.
cv::Mat F_TI;
cv::mulSpectrums(F_I, F_T, F_TI, 0, true);
// Compute the inverse Fourier transform of the cross-correlation,
// dismissing those rows and columns of the cross-correlation
// matrix that would require the template to "roll over" the image.
cv::Size clipped;
clipped.width = I.cols - T.cols;
clipped.height = I.rows - T.rows;
return F_inv(F_TI, clipped);
}
cv::Mat W(const cv::Mat &T)
{
return cv::Mat(T.size(), CV_64F, ONE);
}
cv::Point3f matchTemplate(const cv::Mat &T, const cv::Mat &I)
{
// Compute the optimal size for DFT computing.
cv::Size size = fit(I.size());
//Compute the cross-correlation and normalizing matrix.
cv::Mat C_TI = C(T, I, size);
cv::Mat M_I = C(W(T), I.mul(I), size);
int i_s, j_s;
float r = 0;
int rows = C_TI.rows;
int cols = C_TI.cols;
for (int i = 0; i < rows; i++)
{
for (int j = 0; j < cols; j++)
{
float v = C_TI.at<double>(i, j) / sqrt(M_I.at<double>(i, j));
if (r < v)
{
r = v;
i_s = i;
j_s = j;
}
}
}
return cv::Point3f(j_s, i_s, r);
}
cv::Mat L(const cv::Mat &I)
{
cv::Mat L_I, L_F;
cv::cvtColor(I, L_I, CV_BGR2GRAY);
L_I.convertTo(L_F, CV_64F);
return L_F;
}
int main(int argc, char *argv[])
{
cv::Mat T = cv::imread(argv[1]);
cv::Mat I = cv::imread(argv[2]);
cv::Point3f match = matchTemplate(L(T), L(I));
cv::rectangle(I, cv::Rect(match.x, match.y, T.cols, T.rows), WHITE);
cv::imshow("Template", T);
cv::imshow("Image", I);
cv::waitKey();
return 0;
}
</pre>
Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comtag:blogger.com,1999:blog-1218335813680538098.post-38937726743665138722015-05-21T21:12:00.000-07:002015-05-21T22:05:22.164-07:00Controlling a differential drive robot in Gazebo from ROS<div dir="ltr" style="text-align: left;" trbidi="on"><a href="http://gazebosim.org/" target="_top">Gazebo</a> is a simulation suite targeted at robotics projects, used by DARPA to implement the virtual part of its <a href="http://spectrum.ieee.org/automaton/robotics/robotics-software/osrf-prepares-for-darpa-virtual-robotics-challenge" target="_top">Robotics Challenge</a>. It's a great companion to the <a href="http://www.ros.org/" target="_top">Robot Operating System</a>, supporting <a href="http://gazebosim.org/tutorials?cat=connect_ros" target="_top">several ways</a> to interface with it.<br />
<br />
One recurrent question regarding Gazebo is how to control robots described in <a href="http://sdformat.org/" target="_top">Simulation Description Format (SDF)</a> models from ROS. The more involved case of <a href="http://gazebosim.org/tutorials?tut=ros_urdf&cat=connect_ros" target="_top">porting a URDF model to Gazebo</a> and <a href="http://gazebosim.org/tutorials?tut=ros_control&cat=connect_ros" target="_top">manipulating it through ROS Control</a> interfaces is well documented, but what about a robot that already has a full SDF model available? Isn't it possible to just paste it into a Gazebo workspace and get some ROS topics to manipulate it?<br />
<br />
The tutorial below covers this simpler case. It is based on the SDF model for the <a href="http://www-lar.deis.unibo.it/equipments/p2dx/" target="_top">Pioneer 2DX</a> differential drive robot, which is available from Gazebo's default library. It also assumes ROS and Gazebo are already installed, as well as the necessary <a href="http://gazebosim.org/tutorials?tut=ros_installing&cat=connect_ros" target="_top">integration packages</a> and <a href="http://gazebosim.org/tutorials?tut=ros_gzplugins&cat=connect_ros" target="_top">plugins</a>.<br />
<br />
First, save the SDF world model below to your local filesystem as <code>pioneer2dx_ros.world</code>:<br />
<pre class="prettyprint"><?xml version="1.0" ?>
<sdf version='1.4'>
<world name='default'>
<light name='sun' type='directional'>
<cast_shadows>1</cast_shadows>
<pose>0 0 10 0 -0 0</pose>
<diffuse>0.8 0.8 0.8 1</diffuse>
<specular>0.2 0.2 0.2 1</specular>
<attenuation>
<range>1000</range>
<constant>0.9</constant>
<linear>0.01</linear>
<quadratic>0.001</quadratic>
</attenuation>
<direction>-0.5 0.1 -0.9</direction>
</light>
<model name='ground_plane'>
<static>1</static>
<link name='link'>
<collision name='collision'>
<geometry>
<plane>
<normal>0 0 1</normal>
<size>100 100</size>
</plane>
</geometry>
<surface>
<friction>
<ode>
<mu>100</mu>
<mu2>50</mu2>
</ode>
</friction>
<bounce/>
<contact>
<ode/>
</contact>
</surface>
<max_contacts>10</max_contacts>
</collision>
<visual name='visual'>
<cast_shadows>0</cast_shadows>
<geometry>
<plane>
<normal>0 0 1</normal>
<size>100 100</size>
</plane>
</geometry>
<material>
<script>
<uri>file://media/materials/scripts/gazebo.material</uri>
<name>Gazebo/Grey</name>
</script>
</material>
</visual>
<velocity_decay>
<linear>0</linear>
<angular>0</angular>
</velocity_decay>
<self_collide>0</self_collide>
<kinematic>0</kinematic>
<gravity>1</gravity>
</link>
</model>
<physics type='ode'>
<max_step_size>0.001</max_step_size>
<real_time_factor>1</real_time_factor>
<real_time_update_rate>1000</real_time_update_rate>
<gravity>0 0 -9.8</gravity>
</physics>
<scene>
<ambient>0.4 0.4 0.4 1</ambient>
<background>0.7 0.7 0.7 1</background>
<shadows>1</shadows>
</scene>
<spherical_coordinates>
<surface_model>EARTH_WGS84</surface_model>
<latitude_deg>0</latitude_deg>
<longitude_deg>0</longitude_deg>
<elevation>0</elevation>
<heading_deg>0</heading_deg>
</spherical_coordinates>
<model name='pioneer2dx'>
<link name='chassis'>
<pose>0 0 0.16 0 -0 0</pose>
<inertial>
<mass>5.67</mass>
<inertia>
<ixx>0.07</ixx>
<iyy>0.08</iyy>
<izz>0.1</izz>
<ixy>0</ixy>
<ixz>0</ixz>
<iyz>0</iyz>
</inertia>
</inertial>
<collision name='collision'>
<geometry>
<box>
<size>0.445 0.277 0.17</size>
</box>
</geometry>
<max_contacts>10</max_contacts>
<surface>
<bounce/>
<friction>
<ode/>
</friction>
<contact>
<ode/>
</contact>
</surface>
</collision>
<collision name='castor_collision'>
<pose>-0.2 0 -0.12 0 -0 0</pose>
<geometry>
<sphere>
<radius>0.04</radius>
</sphere>
</geometry>
<surface>
<friction>
<ode>
<mu>0</mu>
<mu2>0</mu2>
<slip1>1</slip1>
<slip2>1</slip2>
</ode>
</friction>
<bounce/>
<contact>
<ode/>
</contact>
</surface>
<max_contacts>10</max_contacts>
</collision>
<visual name='visual'>
<pose>0 0 0.04 0 -0 0</pose>
<geometry>
<mesh>
<uri>model://pioneer2dx/meshes/chassis.dae</uri>
</mesh>
</geometry>
</visual>
<visual name='castor_visual'>
<pose>-0.2 0 -0.12 0 -0 0</pose>
<geometry>
<sphere>
<radius>0.04</radius>
</sphere>
</geometry>
<material>
<script>
<uri>file://media/materials/scripts/gazebo.material</uri>
<name>Gazebo/FlatBlack</name>
</script>
</material>
</visual>
<velocity_decay>
<linear>0</linear>
<angular>0</angular>
</velocity_decay>
<self_collide>0</self_collide>
<kinematic>0</kinematic>
<gravity>1</gravity>
</link>
<link name='right_wheel'>
<pose>0.1 -0.17 0.11 0 1.5707 1.5707</pose>
<inertial>
<mass>1.5</mass>
<inertia>
<ixx>0.0051</ixx>
<iyy>0.0051</iyy>
<izz>0.009</izz>
<ixy>0</ixy>
<ixz>0</ixz>
<iyz>0</iyz>
</inertia>
</inertial>
<collision name='collision'>
<geometry>
<cylinder>
<radius>0.11</radius>
<length>0.05</length>
</cylinder>
</geometry>
<surface>
<friction>
<ode>
<mu>100000</mu>
<mu2>100000</mu2>
<slip1>0</slip1>
<slip2>0</slip2>
</ode>
</friction>
<bounce/>
<contact>
<ode/>
</contact>
</surface>
<max_contacts>10</max_contacts>
</collision>
<visual name='visual'>
<geometry>
<cylinder>
<radius>0.11</radius>
<length>0.05</length>
</cylinder>
</geometry>
<material>
<script>
<uri>file://media/materials/scripts/gazebo.material</uri>
<name>Gazebo/FlatBlack</name>
</script>
</material>
</visual>
<velocity_decay>
<linear>0</linear>
<angular>0</angular>
</velocity_decay>
<self_collide>0</self_collide>
<kinematic>0</kinematic>
<gravity>1</gravity>
</link>
<link name='left_wheel'>
<pose>0.1 0.17 0.11 0 1.5707 1.5707</pose>
<inertial>
<mass>1.5</mass>
<inertia>
<ixx>0.0051</ixx>
<iyy>0.0051</iyy>
<izz>0.009</izz>
<ixy>0</ixy>
<ixz>0</ixz>
<iyz>0</iyz>
</inertia>
</inertial>
<collision name='collision'>
<geometry>
<cylinder>
<radius>0.11</radius>
<length>0.05</length>
</cylinder>
</geometry>
<surface>
<friction>
<ode>
<mu>100000</mu>
<mu2>100000</mu2>
<slip1>0</slip1>
<slip2>0</slip2>
</ode>
</friction>
<bounce/>
<contact>
<ode/>
</contact>
</surface>
<max_contacts>10</max_contacts>
</collision>
<visual name='visual'>
<geometry>
<cylinder>
<radius>0.11</radius>
<length>0.05</length>
</cylinder>
</geometry>
<material>
<script>
<uri>file://media/materials/scripts/gazebo.material</uri>
<name>Gazebo/FlatBlack</name>
</script>
</material>
</visual>
<velocity_decay>
<linear>0</linear>
<angular>0</angular>
</velocity_decay>
<self_collide>0</self_collide>
<kinematic>0</kinematic>
<gravity>1</gravity>
</link>
<joint name='left_wheel_hinge' type='revolute'>
<pose>0 0 -0.03 0 -0 0</pose>
<child>left_wheel</child>
<parent>chassis</parent>
<axis>
<xyz>0 1 0</xyz>
<limit>
<lower>-1e+16</lower>
<upper>1e+16</upper>
</limit>
</axis>
</joint>
<joint name='right_wheel_hinge' type='revolute'>
<pose>0 0 0.03 0 -0 0</pose>
<child>right_wheel</child>
<parent>chassis</parent>
<axis>
<xyz>0 1 0</xyz>
<limit>
<lower>-1e+16</lower>
<upper>1e+16</upper>
</limit>
</axis>
</joint>
<!-- Replaced Gazebo's differential drive plugin with the
ROS-friendly variant -->
<!--
<plugin filename="libDiffDrivePlugin.so" name="diff_drive">
<left_joint>left_wheel_hinge</left_joint>
<right_joint>right_wheel_hinge</right_joint>
<torque>5</torque>
</plugin>
-->
<plugin name='differential_drive_controller'
filename='libgazebo_ros_diff_drive.so'>
<alwaysOn>true</alwaysOn>
<updateRate>100</updateRate>
<leftJoint>left_wheel_hinge</leftJoint>
<rightJoint>right_wheel_hinge</rightJoint>
<wheelSeparation>0.39</wheelSeparation>
<wheelDiameter>0.15</wheelDiameter>
<torque>5</torque>
<commandTopic>cmd_vel</commandTopic>
<odometryTopic>odom</odometryTopic>
<odometryFrame>odom</odometryFrame>
<robotBaseFrame>chassis</robotBaseFrame>
</plugin>
<pose>0 0 0 0 -0 0</pose>
<static>0</static>
</model>
<state world_name='default'>
<sim_time>85 304000000</sim_time>
<real_time>85 849190220</real_time>
<wall_time>1432260579 736436496</wall_time>
<model name='ground_plane'>
<pose>0 0 0 0 -0 0</pose>
<link name='link'>
<pose>0 0 0 0 -0 0</pose>
<velocity>0 0 0 0 -0 0</velocity>
<acceleration>0 0 0 0 -0 0</acceleration>
<wrench>0 0 0 0 -0 0</wrench>
</link>
</model>
<model name='pioneer2dx'>
<pose>-0.103826 0.027961 2e-06 -2e-06 -8e-06 -0.094162</pose>
<link name='chassis'>
<pose>-0.103827 0.027961 0.160002 -2e-06 -8e-06 -0.094162</pose>
<velocity>
-0.000797 0.002229 0.003363 -0.024657 -0.007301 0.0023
</velocity>
<acceleration>0 0 0 0 -0 0</acceleration>
<wrench>0 0 0 0 -0 0</wrench>
</link>
<link name='left_wheel'>
<pose>0.011714 0.187806 0.110002 1.38258 1.57033 2.85912</pose>
<velocity>
-0.000816 0.001565 0.000275 -0.014307 -0.006987 0.002012
</velocity>
<acceleration>0 0 0 0 -0 0</acceleration>
<wrench>0 0 0 0 -0 0</wrench>
</link>
<link name='right_wheel'>
<pose>
-0.020254 -0.150688 0.110002 -1.56886 1.51434 -0.092322
</pose>
<velocity>
-0.000347 0.001559 0.008433 -0.014387 -0.003243 0.000126
</velocity>
<acceleration>0 0 0 0 -0 0</acceleration>
<wrench>0 0 0 0 -0 0</wrench>
</link>
</model>
</state>
<gui fullscreen='0'>
<camera name='user_camera'>
<pose>5 -5 2 0 0.275643 2.35619</pose>
<view_controller>orbit</view_controller>
</camera>
</gui>
</world>
</sdf>
</pre>Open a terminal window and start the ROS middleware by entering:<br />
<pre class="prettyprint">roscore
</pre>Open another terminal window, <code>cd</code> to the folder containing <code>pioneer2dx_ros.world</code> and enter:<br />
<pre class="prettyprint">rosrun gazebo_ros gazebo -file pioneer2dx_ros.world
</pre>You should now have a Gazebo window opened with a Pioneer 2DX placed in the middle of an empty world.<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9j6PAB9QJmd4p3CNx3rIOrUlhSyK8TwLwbTRRf9d2e87iwXtsoHzMoRTWwgOu_v5sBFOLNHz5eLrIsRee1p3UdmiVanx5FhOjOGAs5Ch3ntNdz4CHKMiKgz12zEwmVc-kW0BfwbvYh9g/s1600/snapshot.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9j6PAB9QJmd4p3CNx3rIOrUlhSyK8TwLwbTRRf9d2e87iwXtsoHzMoRTWwgOu_v5sBFOLNHz5eLrIsRee1p3UdmiVanx5FhOjOGAs5Ch3ntNdz4CHKMiKgz12zEwmVc-kW0BfwbvYh9g/s640/snapshot.png" /></a></div>Finally, open a third terminal window (it's the last one, I promise) and check if the differential drive topics have been published:<br />
<pre class="prettyprint">rostopic list
</pre>The following topics should be visible, among others:<br />
<pre class="prettyprint">/pioneer2dx/cmd_vel
/pioneer2dx/odom
</pre>You should now be able to get the robot moving by publishing messages to the <code>/pioneer2dx/cmd_vel</code> topic, e.g.<br />
<pre class="prettyprint">rostopic pub -1 /pioneer2dx/cmd_vel geometry_msgs/Twist \
'{linear: {x: 1.0, y: 0.0, z: 0.0}, angular: {x: 0.0, y: 0.0, z: 1.0}}'
</pre>Should get the robot running in a loop.<br />
</div>Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comtag:blogger.com,1999:blog-1218335813680538098.post-33261468789378023272015-05-17T22:44:00.001-07:002015-05-17T22:45:37.290-07:00How to download all Gazebo models<div dir="ltr" style="text-align: left;" trbidi="on"><br />
<a href="http://gazebosim.org/" target="_top">Gazebo</a> is a wonderful simulation tool widely used in robotics projects. The project includes a <a href="http://models.gazebosim.org/" target="_top">library of 3D models</a> that are automatically downloaded on demand. However this makes the application pause each time a model is used for the first time, which can take a considerable time depending on connection speed and interfere with the flow of work.<br />
<br />
In order to avoid this issue I wrote the below script to do a bulk download of all models in the Gazebo library, then copy them to the local model folder. Simply copy the lines below to a script file on a convenient local folder and run it from a command prompt.<br />
<pre class="prettyprint">#!/bin/sh
# Download all model archive files
wget -l 2 -nc -r "http://models.gazebosim.org/" --accept gz
# This is the folder into which wget downloads the model archives
cd "models.gazebosim.org"
# Extract all model archives
for i in *
do
tar -zvxf "$i/model.tar.gz"
done
# Copy extracted files to the local model folder
cp -vfR * "$HOME/.gazebo/models/"
</pre></div>Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comtag:blogger.com,1999:blog-1218335813680538098.post-2414189602368048662015-04-27T00:44:00.000-07:002015-04-27T00:44:41.775-07:00Building OpenCV's HTML documentation<div dir="ltr" style="text-align: left;" trbidi="on">OpenCV has become the <i>de facto</i> standard vision processing library, but the project remains surprisingly obscure in several areas, particularly in relation to building and installation tasks. For example, it took me a couple <i>hours</i> to discover how to build the project's HTML documentation (though to be fair, I was looking into it concurrently with other work).<br />
<br />
For those also struggling with this, see <a href="http://answers.opencv.org/question/7823/unable-to-build-documentation/" target="_top">this question</a> in the OpenCV Q&A site. In short, it gives the following instructions:<br />
<ol><li>Install <a href="http://sphinx-doc.org/" target="_top">Sphinx</a>;</li>
<li>Run <code>cmake</code> on your OpenCV source base as explained in the <a href="http://docs.opencv.org/doc/tutorials/introduction/linux_install/linux_install.html" target="_top">installation instructions</a>, perhaps adding the <code>-D BUILD_DOCS=ON</code> option for good measure;</li>
<li>Run <code>make html_docs</code> to build the documentation in HTML format.</li>
</ol>If everything goes well the HTML documentation will be built to the subfolder <code>doc/_html/</code> inside the build output folder. If not, check if the output of the <code>cmake</code> call reported items <code>Build Documentation</code> and <code>Sphinx</code> both set to <code>YES</code>. You may also check the cmake script <code>cmake/OpenCVDetectPython.cmake</code> inside your OpenCV source base for potential problems.<br />
<br />
</div>Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comtag:blogger.com,1999:blog-1218335813680538098.post-20555523701400047462015-04-08T22:01:00.001-07:002015-04-08T22:01:56.250-07:00Internet sharing for Intel Edison on Linux<div dir="ltr" style="text-align: left;" trbidi="on">Recently I got hold of an <a href="http://www.intel.com/content/www/us/en/do-it-yourself/edison.html" target="_top">Intel Edison</a> with accompanying <a href="https://software.intel.com/en-us/articles/intel-edison-arduino-expansion-board-assembly" target="_top">Arduino Expansion Board</a>. The awkward development setup process (which requires two separate USB cable connections to a host computer) didn't make for a stellar first impression, but the possibilities of a full x86-64 computer that fits in a matchbox are intriguing. Of course, the fact both Linux and <a href="http://wiki.ros.org/wiki/edison"http://wiki.ros.org/wiki/edison>ROS</a> are more-or-less readily available for the platform made the whole proposition a lot more interesting.<br />
<br />
After flashing Edison with <a href="http://www.emutexlabs.com/ubilinux" target="_top">ubilinux</a> the first order of business was getting the thing connected to the Internet. Edison includes an on-board wi-fi card and can simulate an ethernet port through its USB connections to the host computer, but bureaucratic questions would make a direct connection to my lab's network difficult. So I set up my notebook (Edison's host computer) as a local gateway, enabling Internet sharing through the virtual <code>usb0</code> interface.<br />
<br />
The procedure documented below was developed on a <a href="http://www.kubuntu.org/news/kubuntu-14.04" target="_top">Kubuntu 14.04</a> host and ubilinux Edison unit, but it should be adaptable to other Linux distributions. It is largely based on a <a href="http://machineawakening.blogspot.jp/2013/07/setup-wireless-access-point-with-mint.html" target="_top">previous post</a> on wireless Internet sharing; in fact that may be a better option, if unlike me you have a wireless interface to spare.<br />
<br />
First open a command prompt and start a root session with <code>su</code>, then follow the steps below:<br />
<br />
<b>Step 1: Install Dnsmasq</b><br />
<br />
<a href="http://www.thekelleys.org.uk/dnsmasq/doc.html" target="_top">Dnsmasq</a> is a tool for setting up small networks. Install it from the command line by typing:<br />
<pre class="prettyprint">apt-get install dnsmasq
</pre>As it is the <code>dnsmasq</code> service will be automatically started when the computer boots, but we need better control than that. So change the default settings to manual start:<br />
<pre class="prettyprint">update-rc.d -f dnsmasq remove</pre><br />
<b>Step 2: Configuration Files</b><br />
<br />
Create or update the following config file with the contents below:<br />
<br />
<code>/etc/dnsmasq.conf</code><br />
<pre class="prettyprint"># disables dnsmasq reading any other files like /etc/resolv.conf for nameservers
no-resolv
# Interface to bind to
interface=usb0
# Specify starting_range,end_range,lease_time
dhcp-range=usb0,192.168.100.100,192.168.100.199,12h
# dns addresses to send to the clients
server=8.8.8.8
server=8.8.4.4
</pre></div><br />
<b>Step 3: Interface Scripts</b><br />
<br />
Add the scripts below to your system:<br />
<br />
<code>/usr/local/bin/usb0start.sh</code><br />
<pre class="prettyprint">#!/bin/sh
ETH=eth0
USB=usb0
IP=192.168.100.99
MASK=255.255.255.0
# Enable IP forwarding
echo 1 > /proc/sys/net/ipv4/ip_forward
# Setup USB interface
ifconfig $USB up $IP netmask $MASK
# Start dnsmasq
/etc/init.d/dnsmasq start
# Enable NAT
iptables --flush
iptables --table nat --flush
iptables --delete-chain
iptables --table nat --delete-chain
iptables --table nat --append POSTROUTING --out-interface $ETH -j MASQUERADE
iptables --append FORWARD --in-interface $USB -j ACCEPT
</pre><br />
<code>/usr/local/bin/usb0stop.sh</code><br />
<pre class="prettyprint">#!/bin/bash
USB=usb0
# Stops DNS making service
/etc/init.d/dnsmasq stop
# Asked nice the first time...
killall dnsmasq
ifconfig $USB down
</pre>Make sure to create the scripts from the root account, so they'll have the right ownership. Also don't forget to give them running permission:<br />
<pre class="prettyprint">chmod +x /usr/local/bin/usb0start.sh
chmod +x /usr/local/bin/usb0stop.sh
</pre><br />
<b>Step 4: Edison Configuration</b><br />
<br />
Connect Edison to the host computer and <a href="https://software.intel.com/en-us/articles/getting-started-with-the-intel-edison-board-on-linux#terminal" target="_top">start a remote session</a>. Change the following configuration file as below:<br />
<br />
<code>/etc/network/interfaces</code><br />
<pre class="prettyprint"># Disable auto connection, enable DHCP
iface usb0 inet dhcp
#auto usb0
#iface usb0 inet static
# address 192.168.2.15
# netmask 255.255.255.0
</pre><br />
<b>Usage</b><br />
<br />
After performing the steps above, you should be able to start the local gateway by running the <code>usb0start.sh</code> script:<br />
<pre class="prettyprint">sudo /usr/local/bin/usb0start.sh</pre><br />
Then on the Edison environment, start the <code>usb0</code> interface:<br />
<pre class="prettyprint">ifup usb0</pre><br />
After you're done with Edison, the gateway can be deactivated by running the <code>usb0stop.sh</code> script:<br />
<pre class="prettyprint">sudo /usr/local/bin/usb0stop.sh</pre><br />
<b>Troubleshooting</b><br />
<br />
If Edison cannot lease an IP from the host computer or see IP addresses beyond the gateway, you can enable logging on Dnsmasq by adding the following lines to the <code>/etc/dnsmasq.conf</code> file:<br />
<pre class="prettyprint">log-queries
log-dhcp
</pre><br />
Then use <code>tail -f /var/log/syslog</code> to check for messages.<br />
<br />
On Kubuntu the network manager tried to setup the <code>usb0</code> interface using <code>eth0</code>'s configuration every time it was turned on, messing the DHCP service. This was solved by explicitly binding the configuration to <code>eth0</code> on the network manager's GUI.Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comtag:blogger.com,1999:blog-1218335813680538098.post-89383762931107450492015-04-06T20:36:00.000-07:002015-05-21T20:09:55.036-07:00ROS and my first pull request<div dir="ltr" style="text-align: left;" trbidi="on">I had been putting it off for a long time, but a couple weeks ago I finally decided to bite the bullet and port my research code to <a href="http://www.ros.org/" target="_top">ROS</a>. After a bit of a rough start reinstalling my OS from scratch – I had recently upgraded my notebook to Kubuntu 14.10, but the latest ROS distribution (Indigo) wouldn't install on anything past 14.04 – and coming to terms with the <a href="http://wiki.ros.org/catkin" target="_top">catkin</a> build system, I quickly fell in love with its modular approach to software development.<br />
<br />
One of the many little conveniences ROS offers are <a href="http://wiki.ros.org/Parameter%20Server" target="_top">parameters</a>, which among other things greatly simplify management of program arguments. The <a href="http://wiki.ros.org/roscpp" target="_top">ROS C++ client library</a> provides several ways to retrieve parameter values, and optionally provide defaults for when they are not defined. One of those ways is the <code>ros::param::param</code> template function, which can be used to write initialization code like this:<br />
<pre class="prettyprint">std::string path;
std::string format;
double fps;
param("~path", path, "video.mpg");
param("~format", format, "MPEG");
param("~fps", fps, 30.0);
VideoRecorder recorder(path, format, fps);
</pre>However I disliked having to declare variables to get to the parameter values, just to pass them straight to a constructor or function call and then disregard them. So I wrote an alternative version of the template function that retrieved the value as a return instead of an output argument:<br />
<pre class="prettyprint">template<class T> T param(const std::string& name, const T& fallback)
{
T value;
param(name, value, fallback);
return value;
}
</pre>So I could rewrite the code above as:<br />
<pre class="prettyprint">VideoRecorder recorder(param<std::string>("~path", "video.mpg"),
param<std::string>("~format", "MPEG"),
param<double>("~fps", 30.0));
</pre>At first I had separate copies of this function on each of my ROS projects, but soon I realized it would be great if I could contribute it to the official <code>roscpp</code> code base. After a bit of <a href="https://github.com/ros-infrastructure/rep/issues/97" target="_top">stumbling</a> <a href="https://github.com/ros/ros_comm/issues/591" target="_top">about</a> looking for the right place to propose it, I ended up submitting the change myself as a <a href="https://github.com/ros/ros_comm/pull/592" target="_top">pull request</a>. Overall the <a href="https://help.github.com/articles/using-pull-requests/" target="_top">process</a> went very smoothly, though there are some tips I wish I had been given before I started:<br />
<ul><li>You'll want to compile and run tests on the code you intend to submit (believe me, even a change as small as the one above was not without a couple coding errors), so remember to create a <a href="http://wiki.ros.org/catkin/Tutorials/create_a_workspace" target="_top">catkin workspace</a> into which to clone the forked repository;</li>
<li>You may think that your change is trivial and will be done in a single commit, but it's likely your submission will go through several revisions before it's accepted. To keep things organized, create a new branch before starting your work;</li>
<li>Beware of line-ending whitespace – there are a few lurking around in source files, and if your editor automatically removes them you'll end with an unnecessarily large (and likely unacceptable) diff;</li>
<li>That said if you do mess up, don't bother trying to amend the initial commit – simply submit additional commits with the required changes or reversals (this is where having a separate branch may come in handy).</li>
</ul>Open communities are not always noted for being welcoming to beginners, even less so in technical circles. But I have nothing but praise for the <a href="https://github.com/ros/ros_comm/">ros_comm</a> community and the fairly friendly and constructive exchange we had. Keep up the good work people!<br />
</div>Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comtag:blogger.com,1999:blog-1218335813680538098.post-20585961638847106032013-07-13T18:49:00.001-07:002017-06-25T06:26:11.796-07:00Setup a Wireless Access Point with Linux Mint 15 (Olivia)<div dir="ltr" style="text-align: justify;" trbidi="on">In these days of tablets, smartphones and other "connected" devices, the ability to share a fast wired Internet connection via wi-fi can be very useful, specially at home or small offices. Wireless routers aren't very expensive and are simple enough to install – yet I have always thought they're kind of wasteful, considering most computers today ship with wi-fi interfaces, or can be fitted with USB cards that sell for pennies.<br />
<br />
With Linux, wi-fi interfaces can be configured to operate in <i>master mode,</i> so they can accept client connections just like a dedicated router would. This is a huge improvement over "ad-hoc" wi-fi network sharing, because:<ol style="text-align: justify;"><li>Many mobile devices do not connect to "ad-hoc" access points;</li>
<li>The WEP authentication standard used on those connections is known to be flawed, in contrast to the newer WPA standard used in "master mode" connections.</li>
</ol>Today I updated my desktop to the latest Linux Mint 15 (Olivia), and as I went about configuring a Wireless Access Point (WAP) on it for my other devices, I took the time to document my changes to the system, so I can more easily reproduce them in future upgrades – and hopefully, also help others to setup a WAP with Mint.<br />
<br />
The procedure I came up with was largely inspired by <a href="http://nims11.wordpress.com/2013/05/22/using-hostapd-with-dnsmasq-to-create-virtual-wifi-access-point-in-linux/">this blog post</a>. It assumes the existence of a wired interface <code>eth0</code> (which provides access to the Internet) and a <a href="http://wireless.kernel.org/en/developers/Documentation/mac80211">mac80211</a>-compatible wireless interface <code>wlan0</code> (which will be configured to accept client connections in "master mode").<br />
<br />
First open a command prompt and start a root session with <span style="font-family: "Courier New",Courier,monospace;">su</span>, then follow the steps below:<br />
<br />
<b>Step 1: Install Applications</b><br />
<br />
Type the command below to install the required services:<br />
<pre class="prettyprint">apt-get install hostapd dnsmasq
</pre>As it is the services will be automatically started when the computer boots, but we need better control than that. So change the default settings to manual start:<br />
<pre class="prettyprint">update-rc.d -f hostapd remove
update-rc.d -f dnsmasq remove
</pre><br />
<b>Step 2: Configuration Files</b><br />
<br />
Create or update the following config files with the contents below:<br />
<br />
<code>/etc/hostapd/hostapd.conf</code><br />
<pre class="prettyprint">interface=wlan0
driver=nl80211
ssid=hotspot # Your WAP name, change it to something more unique
hw_mode=g
channel=6 # You may want to change this if the channel is too crowded
wpa=1
wpa_passphrase=hotspot_password # Password for clients
wpa_key_mgmt=WPA-PSK
wpa_pairwise=TKIP CCMP
wpa_ptk_rekey=600
</pre><br />
<code>/etc/dnsmasq.conf</code><br />
<pre class="prettyprint"># disables dnsmasq reading any other files like /etc/resolv.conf for nameservers
no-resolv
# Interface to bind to
interface=wlan0
# Specify starting_range,end_range,lease_time
dhcp-range=10.0.0.3,10.0.0.20,12h
# dns addresses to send to the clients
server=8.8.8.8
server=8.8.4.4
</pre><br />
<b>Step 3: Interface Scripts</b><br />
<br />
Add the scripts below to your system:<br />
<br />
<code>/etc/network/if-up.d/wapstart</code><br />
<pre class="prettyprint">#!/bin/sh
WIRE=eth0
WIFI=wlan0
# Only run script for wired interface
if [ ! "$IFACE" = "$WIRE" ]
then
exit 0
fi
# Setup wireless interface
ifconfig $WIFI up 10.0.0.1 netmask 255.255.255.0
# Start dnsmasq
/etc/init.d/dnsmasq start
#Enable NAT
iptables --flush
iptables --table nat --flush
iptables --delete-chain
iptables --table nat --delete-chain
iptables --table nat --append POSTROUTING --out-interface $WIRE -j MASQUERADE
iptables --append FORWARD --in-interface $WIFI -j ACCEPT
# Start the Wireless Access Point service
/etc/init.d/hostapd start
exit 0
</pre><br />
<code>/etc/network/if-post-down.d/wapstop</code><br />
<pre class="prettyprint">#!/bin/bash
WIRE=eth0
WIFI=wlan0
# Only run script for wired interface
if [ ! "$IFACE" = "$WIRE" ]
then
exit 0
fi
# Stops Wireless Access Point services
/etc/init.d/hostapd stop
/etc/init.d/dnsmasq stop
# Asked nice the first time...
killall dnsmasq
killall hostapd
ifconfig $WIFI down
</pre>Make sure to create the scripts from the root account, so they'll have the right ownership. Also don't forget to give them running permission:<br />
<pre class="prettyprint">chmod +x /etc/network/if-up.d/wapstart
chmod +x /etc/network/if-post-down.d/wapstop
</pre><br />
<b>Usage</b><br />
<br />
After performing the steps above, you should be able to start the WAP by stopping and restarting the wired interface:<br />
<pre class="prettyprint">ifconfig eth0 down
ifconfig eth0 up
</pre>From now on it will also be automatically started at boot time.<br />
<br />
<b>Troubleshooting</b><br />
<br />
If your devices can see and connet to the wireless hotspot, but they cannot access the Internet, enter the command below:<br />
<pre class="prettyprint">sysctl -w net.ipv4.ip_forward=1
</pre>This will update kernel settings to enable IP forwarding, and only has to be issued once.<br />
<br />
Depending on your distribution, <code>hostapd</code> may require that the wireless interface be either turned "on" in the network manager applet (e.g. Mint 15), <i>or</i> excluded from Network Manager control (e.g. Kubuntu 14.04). In the second case add the following to <code>/etc/NetworkManager/NetworkManager.conf</code>:<br />
<pre class="prettyprint">[keyfile]
unmanaged-devices=mac:mac_address_of_wireless_interface
</pre>Also check <code>/etc/init.d/hostapd</code> for whether the variable <code>DAEMON_CONF</code> is set; if not, set it to the path of the config file:<br />
<pre class="prettyprint">DAEMON_CONF=/etc/hostapd/hostapd.conf
</pre>If nothing else works, you can try to find the cause by running <code>hostapd</code> directly instead of in service mode, with extra logging options enabled. For example:<br />
<pre class="prettyprint">hostapd -d /etc/hostapd/hostapd.conf
</pre></div>Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comtag:blogger.com,1999:blog-1218335813680538098.post-40937925900805576202013-03-19T07:52:00.001-07:002013-03-19T07:52:59.157-07:00John Nash Vindicated?<div dir="ltr" style="text-align: left;" trbidi="on">
Kaspersky Lab's Global Research & Analysis Team (GReAT) reports some <a href="https://www.securelist.com/en/blog/208194129/The_MiniDuke_Mystery_PDF_0_day_Government_Spy_Assembler_Micro_Backdoor">unsettling details on the MiniDuke malware</a>:<br /><br /><i>(...) If the target system meets the pre-defined requirements, the malware will use Twitter (unbeknownst to the user) and start looking for specific tweets from pre-made accounts. These accounts were created by MiniDuke’s Command and Control (C2) operators and the tweets maintain specific tags labeling encrypted URLs for the backdoors.<br /><br />Once the infected system locates the C2, it receives encrypted backdoors that are obfuscated within GIF files and disguised as pictures that appear on a victim’s machine.<br /><br />Once they are downloaded to the machine, they can fetch a larger backdoor which carries out the cyberespionage activities, through functions such as copy file, move file, remove file, make directory, kill process and of course, download and execute new malware and lateral movement tools.</i><br /><br />An espionage system that talks to his masters via hidden messages in public media? I guess we all owe <a href="http://en.wikipedia.org/wiki/A_Beautiful_Mind_(film)">Mr. Nash</a> an apology then...</div>
Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comtag:blogger.com,1999:blog-1218335813680538098.post-81846327836931358332013-03-18T12:19:00.000-07:002013-03-18T12:23:43.791-07:00Straight out of a Jetsons episode, the future of robotic manufacture<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both; text-align: center;">
<a href="http://www.technologyreview.com/sites/default/files/styles/body_embed/public/legacy/rodney.brooksx616.jpg?itok=kODlWPIu" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://www.technologyreview.com/sites/default/files/styles/body_embed/public/legacy/rodney.brooksx616.jpg?itok=kODlWPIu" /></a></div>
<br />
<div style="text-align: justify;">
With its vaguely humanoid frame and cartoonish screen-drawn eyes, <a href="http://www.rethinkrobotics.com/index.php/products/baxter/">Baxter</a> looks like an early-20th century artist's impression of a robot. And indeed, in many ways it is closer from those old ideals of pervasive robotics than the practical restrictions we have come to expect from industrial robots. Will Knight <a href="http://www.technologyreview.com/news/429248/this-robot-could-transform-manufacturing/">writes</a>:</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<i>Baxter is the first of a new generation of smarter, more adaptive industrial robots. Conventional industrial robots are expensive to program, incapable of handling even small deviations in their environment, and so dangerous that they have to be physically separated from human workers by cages. (...) Baxter, however, can be programmed more easily than a Tivo and can deftly respond to a toppled-over part or shifted table. And it is so safe that Baxter’s developer, <a href="http://www.rethinkrobotics.com/">Rethink Robotics</a>, which loaned Baxter to Vanguard Plastics, believes it can work seamlessly alongside its human coworkers.</i></div>
</div>
Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comtag:blogger.com,1999:blog-1218335813680538098.post-10735492996495499212013-01-19T06:55:00.001-08:002013-03-18T10:38:13.136-07:00Genkou Youshi Template for OpenOffice / LibreOffice<div dir="ltr" style="text-align: left;" trbidi="on">
Writing assignments on <i><a href="http://en.wikipedia.org/wiki/Genkoo_yooshi">genkou youshi</a></i> are a staple of Japanese language classes. Tradition and the need to practice Kanji writing often dictate that such works be handwritten, but there's no reason one couldn't first compose the text on a computer (where editing, revision and indeed writing itself are much easier) and only reach to paper and pencil when the final version is ready.<br />
<br />
Experience has taught me that hand-copying texts from a computer screen is much quicker and less error-prone when both digital document and paper share the same layout, and of course it's best if I can print <i>genkou youshi</i> sheets at my own leisure, rather than go out and buy them (not many stationery shops this side of the world have them, anyway). Searching the web it is relatively easy to find a <a href="http://www.jpf.org.uk/language/download/genkouyooshi.doc"><i>genkou youshi</i> template for Microsoft Word</a>, but Open Document Format equivalents are harder to come by – and while it's possible to use the Word templates on OpenOffice / LibreOffice, rendering problems such as misaligned grids and hidden hanging punctuation are often present, which is kind of frustrating for a die-hard Linux user like me.<br />
<br />
Today as I sit down to try and create a <i>genkou youshi</i> template on LibreOffice, I decided to try and look it out one last time – and there I found a <a href="http://templates.services.openoffice.org/en/node/1121"><i>genkou youshi</i> Open Template</a> on the <a href="http://templates.services.openoffice.org/en">OpenOffice Templates</a> web site. I'm kind of baffled as to how I didn't come across it earlier (apparently it was uploaded in 2009), so I thought I'd link to it here, so others who might also be looking for it have a better chance of finding it. So there.</div>
Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comtag:blogger.com,1999:blog-1218335813680538098.post-19454988020927607022013-01-04T07:58:00.000-08:002013-01-19T05:24:25.689-08:00batcher - simple shell wrappers for Python<div dir="ltr" style="text-align: left;" trbidi="on">One of my favorite Linux features is how repetitive tasks can be automated via shell scripts. Unfortunately, the Bourne shell language and its variations (e.g. Bash) are bogged down by a number of idiosyncrasies – such as the nearly constant need to enclose variable references in quotes, the awkwardness of conditional constructs, the difficulties involved in performing simple arithmetics and so on – which needlessly complicate development. On the other hand, the ability to directly invoke user shell commands, scripts and command-line applications is a convenience seldom matched by alternatives, which explains why we mostly put up with it.<br />
<br />
A while ago I wanted to write a relatively complex automation script, and not looking forward to doing the required text processing in Bash, decided instead to try my luck with Python, using <span style="font-family: Courier New, Courier, monospace;">Popen</span> objects to invoke the needed command-line applications and collect their outputs. In order to make this task simpler, I came up with the function below:<br />
<pre class="prettyprint">def batch(*args, **options):
options.setdefault('stdout', PIPE)
options.setdefault('stderr', STDOUT)
process = Popen(args, **options)
return process.stdout
</pre>This allowed me to interface to shell tools in a straightforward way, for example:<br />
<pre class="prettyprint">for line in batch('git', 'status', '-s', cwd=path):
# Do some line-oriented processing with the command's output
</pre>While it was simple and worked well enough, this interface surely had room for improvement. It bothered me that I had no access to the <span style="font-family: Courier New, Courier, monospace;">Popen</span> object driving the shell process, but only to its <span style="font-family: Courier New, Courier, monospace;">stdout</span> output object; I also felt it was wasteful that the process would run to completion even if control broke off the loop before it finished. Finally, on an aesthetic note, I thought it would be nice if there was some "automagic" way to turn command names into callables, so I could e.g. do <span style="font-family: Courier New, Courier, monospace;">git('status', '-s')</span> instead of <span style="font-family: Courier New, Courier, monospace;">batch('git', 'status', '-s')</span>.<br />
<br />
After some tinkering with Python's reflection API, I came up with the code below:<br />
<pre class="prettyprint">#! /usr/bin/env python
#coding=utf-8
# batcher.py
# Dynamic interfaces to shell commands
from subprocess import Popen, PIPE, STDOUT
from sys import modules
class batch(Popen):
r'''A running shell command or script.
'''
def __init__(self, command, args, options):
args = (command,) + args
options.setdefault('stdout', PIPE)
options.setdefault('stderr', STDOUT)
Popen.__init__(self, args, **options)
self.daemon = options.get('daemon', True)
def __del__(self):
if self.daemon:
self.close()
else:
Popen.__del__(self)
def __enter__(self):
return self
def __exit__(self, exc_type, exc_value, traceback):
self.close()
def __iter__(self):
return self
def close(self):
if self.returncode != None:
return
try:
self.kill()
except:
pass
def next(self):
return self.stdout.next().rstrip()
class batcher_module(object):
r'''An extension to the default module object, which creates new
command-calling functions as they are imported.
'''
def __init__(self, module):
self.module = module
def __getattr__(self, name):
try:
return getattr(self.module, name)
except AttributeError:
pass
def batcher(*args, **options):
return batch(name, args, options)
setattr(self.module, name, batcher)
return batcher
modules[__name__] = batcher_module(modules[__name__])
</pre>When a symbol is imported from the <span style="font-family: Courier New, Courier, monospace;">batcher</span> module, the <span style="font-family: Courier New, Courier, monospace;">batcher_module</span> object first checks whether it already exists. If it doesn't (which is most likely), the symbol is created and bound to a function that will run the correspondingly-named shell command when called. That function works by instantiating an object of the <span style="font-family: Courier New, Courier, monospace;">batch</span> class, which inherits from <span style="font-family: Courier New, Courier, monospace;">Popen</span> and adds the following customizations and extensions:<br />
<ul style="text-align: left;"><li>By default both standard and error outputs are collected in the <span style="font-family: Courier New, Courier, monospace;">stdout</span> attribute. This can be changed at instantiation time using the named arguments <span style="font-family: Courier New, Courier, monospace;">stdout</span> and <span style="font-family: Courier New, Courier, monospace;">stderr</span>;</li>
<li>Whereas <span style="font-family: Courier New, Courier, monospace;">Popen</span> objects try to remain alive for as long as the underlying process is active, <span style="font-family: Courier New, Courier, monospace;">batch</span> objects work the other way around by killing the process (if not yet finished) when they're selected for garbage collection (the original behavior can be restored by passing the <span style="font-family: Courier New, Courier, monospace;">daemon</span> named argument with a value of <span style="font-family: Courier New, Courier, monospace;">False</span>);</li>
<li><span style="font-family: Courier New, Courier, monospace;">batch</span> objects are iterable, returning the next (right-trimmed) line of output at each iteration;</li>
<li><span style="font-family: Courier New, Courier, monospace;">batch</span> objects are also their own context managers, killing the underlying process (if not yet finished) when the context is exited;</li>
<li>Finally, <span style="font-family: Courier New, Courier, monospace;">batch</span> objects implement the <span style="font-family: Courier New, Courier, monospace;">close()</span> method, which kills the underlying process if it hasn't yet finished, and does nothing otherwise.</li>
</ul>Using the <span style="font-family: Courier New, Courier, monospace;">batcher</span> module makes it possible to write Python scripts that seamlessly interface with shell applications, for example:<br />
<pre class="prettyprint">#! /usr/bin/env python
#coding=utf-8
# repo_dirty.py - scans a repo base for git projects containing non-commited changes
from re import search
from batcher import git, repo
def gitdirty(path):
for line in git('status', cwd=path):
if search(r'(On branch master)|(nothing to commit)', line) != None:
print 'Project "%s" is dirty' % path
return True
return False
def repodirty():
projects = 0
dirty = 0
for path in repo('forall', '-c', 'pwd'):
projects += 1
dirty += 1 if gitdirty(path) else 0
print 'Finished checking dirty projects'
print 'Total projects checked: %d' % projects
print 'Dirty projects: %d' % dirty
def main():
repodirty()
if __name__ == '__main__':
main()
</pre>For the past few days I have been using this interface to rewrite some of my own shell scripts in Python, and I am startled with the results. So far it worked without a hiccup; being able to use Python on shell automation duties brought great improvements to performance and reliability, as well as my own personal satisfaction with the resulting code. Between Python's power and this newfound seamless integration to the shell environment, I now see seldom reason to ever bother writing a shell script again.</div>Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comtag:blogger.com,1999:blog-1218335813680538098.post-77366693612136067702012-04-03T09:44:00.000-07:002012-04-03T09:44:17.858-07:00Boot2Gecko and The Infinite Indifference<i>Vision Mobile</i>'s Stijn Schuermans <a href="http://www.visionmobile.com/blog/2012/03/boot2gecko-can-the-new-html5-champion-succeed-where-webos-failed/">reports</a> the Boot2Gecko demonstration at MWC 2012, and discusses the system's perspectives on the current environment. His point is that Telefonica (and possibly later other carriers) could employ it as the base for a commodity application ecosystem, that would wrestle control of the market – and hopefully revenues – back from Apple and Google.<br />
<br />
Of course, mobile operators already have a web technologies-based platform with which to fight Apple and Google. It's called the Wholesale Application Community (WAC): it's got a Javascript API, a W3C-compatible application packaging model, an infrastructure for operator-local app stores, the support of a string of carriers, model manufacturers and IT companies – and is going nowhere but down.<br />
<br />
Did you know that Android is, on paper, the product of the Open Handset Alliance? That's a consortium of more than 80 companies – yet Google seems to be the only one actually working for its progress, while the others do little more than port the software to half-baked reference hardware and slap their logos on top of the things.<br />
<br />
Technology is not the problem, lack of compromise is. Players in the mobile industry will gather to draw up (and sometimes even implement) standards all the time; getting any one of them to promote its use afterwards is another matter entirely. The whole mobile industry is terribly inertial: that's how they got driven off to the borders of their own market by Apple and Google to begin with. That's why WAP languished for years until being crushed under the rise of the mobile web, why WAC is stillborn, and why IMS will ultimately prove irrelevant.<br />
<br />
The reason Boot2Gecko just might succeed is not technology, Telefonica's support or the increasing marginalization of carriers. The best thing it's got going for it is Mozilla's role as the driving force behind the platform. Just as with Android and Google, they've got the will to keep promoting it long after other involved parties have lost interest (which if experience is any guide, will happen within seconds of the 1.0 release).<br />
<br />
If Mozilla can attract developer mindshare and drag the carriers and product manufacturers to provide consistent support, then Boot2Gecko may realize that vision of a web application platform which goes back all the way to WAP. Of course history is against it, but it's the best chance we've got so far.Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comtag:blogger.com,1999:blog-1218335813680538098.post-61568732948639393122012-02-22T13:36:00.000-08:002012-02-22T13:40:35.130-08:00A Python Implementation of The Log-Polar Transform, Redux<br />
<table><tbody>
<tr><td><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDh7xgZO4HnICrfMNLXIP2S2Em-QPNQi4QeKj09GYlw_QK3X_XYGUogRGU4_B24GNS2u2wa_LEGvJyFzLGPOkwBW5hxZGN2oU2NXpUc3feHPBzpQcuw_ah3luohfB49dhGnW7lEgVHXHk/s1600/cartesian.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDh7xgZO4HnICrfMNLXIP2S2Em-QPNQi4QeKj09GYlw_QK3X_XYGUogRGU4_B24GNS2u2wa_LEGvJyFzLGPOkwBW5hxZGN2oU2NXpUc3feHPBzpQcuw_ah3luohfB49dhGnW7lEgVHXHk/s1600/cartesian.png" /></a></td><td><br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgtW5PLA7tCukn-3EvU_drF_d8lQpH-e03I9AQH1EOzCsCTyewn6YU-yv92xhYX7N3dw_vCuKQ2yqwp-1NMERnG8tYfNxVBU8AQzyWRMuEUaVS9QTzr7tFMHcXurCIjZR2qeyXfdjL4ozc/s1600/logpolar_fancy.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgtW5PLA7tCukn-3EvU_drF_d8lQpH-e03I9AQH1EOzCsCTyewn6YU-yv92xhYX7N3dw_vCuKQ2yqwp-1NMERnG8tYfNxVBU8AQzyWRMuEUaVS9QTzr7tFMHcXurCIjZR2qeyXfdjL4ozc/s1600/logpolar_fancy.png" /></a></td></tr>
</tbody></table><br />
One of my <a href="http://machineawakening.blogspot.com/2009/07/python-implementation-of-log-polar.html">first posts to this blog</a> presented a Python implementation of the <a href="http://en.wikipedia.org/wiki/Log-polar_coordinates">log-polar transform</a>. Back then I was just beginning to learn <a href="http://numpy.scipy.org/">NumPy</a>, and didn't know many of its more useful tricks; in particular, I had no domain of <a href="http://www.scipy.org/Tentative_NumPy_Tutorial#head-0dffc419afa7d77d51062d40d2d84143db8216c2">fancy indexing</a>, and did not know that it could be used to avoid the need for time-consuming Python loops. Besides that, I later realized that the algorithm I chose to implement, taken from <a href="http://www.vuse.vanderbilt.edu/~rap2/papers/oncomplp.pdf">an article</a> I had read at the time, was unnecessarily complicated.<br />
<br />
A couple of months ago I was considering the use of the log-polar transform for another project, and thought I should go back and rewrite my early implementation. I wanted to make use of the NumPy expertise I've earned since then, but also clean up my early blunder and present a simpler algorithm, that could be easily ported to other languages. The results are available below, and also for <a href="https://sites.google.com/site/machineawakeningsvault/home/Log%20Polar%20Transform%20Redux.zip">download</a> in a nice package, with test code and thoroughly commented sources.<br />
<br />
The first listing presents a "naive" implementation, which relies on Python loops:<br />
<pre class="prettyprint">def logpolar_naive(image, i_0, j_0, p_n=None, t_n=None):
(i_n, j_n) = image.shape[:2]
i_c = max(i_0, i_n - i_0)
j_c = max(j_0, j_n - j_0)
d_c = (i_c ** 2 + j_c ** 2) ** 0.5
if p_n == None:
p_n = int(ceil(d_c))
if t_n == None:
t_n = j_n
p_s = log(d_c) / p_n
t_s = 2.0 * pi / t_n
transformed = zeros((p_n, t_n) + image.shape[2:], dtype=image.dtype)
for p in range(0, p_n):
p_exp = exp(p * p_s)
for t in range(0, t_n):
t_rad = t * t_s
i = int(i_0 + p_exp * sin(t_rad))
j = int(j_0 + p_exp * cos(t_rad))
if 0 <= i < i_n and 0 <= j < j_n:
transformed[p, t] = image[i, j]
return transformed
</pre>The second version constructs a pair of NumPy fancy indices first – cached for later reference – and then uses them to perform the transform: <br />
<pre class="prettyprint">_transforms = {}
def _get_transform(i_0, j_0, i_n, j_n, p_n, t_n, p_s, t_s):
transform = _transforms.get((i_0, j_0, i_n, j_n, p_n, t_n))
if transform == None:
i_k = []
j_k = []
p_k = []
t_k = []
for p in range(0, p_n):
p_exp = exp(p * p_s)
for t in range(0, t_n):
t_rad = t * t_s
i = int(i_0 + p_exp * sin(t_rad))
j = int(j_0 + p_exp * cos(t_rad))
if 0 <= i < i_n and 0 <= j < j_n:
i_k.append(i)
j_k.append(j)
p_k.append(p)
t_k.append(t)
transform = ((array(p_k), array(t_k)), (array(i_k), array(j_k)))
_transforms[i_0, j_0, i_n, j_n, p_n, t_n] = transform
return transform
def logpolar_fancy(image, i_0, j_0, p_n=None, t_n=None):
(i_n, j_n) = image.shape[:2]
i_c = max(i_0, i_n - i_0)
j_c = max(j_0, j_n - j_0)
d_c = (i_c ** 2 + j_c ** 2) ** 0.5
if p_n == None:
p_n = int(ceil(d_c))
if t_n == None:
t_n = j_n
p_s = log(d_c) / p_n
t_s = 2.0 * pi / t_n
(pt, ij) = _get_transform(i_0, j_0, i_n, j_n, p_n, t_n, p_s, t_s)
transformed = zeros((p_n, t_n) + image.shape[2:], dtype=image.dtype)
transformed[pt] = image[ij]
return transformed
</pre>Both implementations are based on a "reverse transform" algorithm: rather than iterate over the Cartesian image calculating where each pixel fits in the transform (which resulted in all sorts of complications), iterate over the <i>transform's</i> cells and calculate which, if any, pixel from the Cartesian image should be put there.<br />
<br />
According to my tests, the "fancy" version using a cached pair of indices runs about 10 times faster than the "naive" version – but even when it has to construct the indices before using them, it's still slightly faster.Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comtag:blogger.com,1999:blog-1218335813680538098.post-19259722707653303752012-02-11T12:43:00.000-08:002012-02-11T12:46:34.102-08:00On the Difference Between Computers and Computing MachinesA recent discussion on <a href="http://tech.groups.yahoo.com/group/Robitron/">Robtiron</a> has cast doubt on whether Alan Turing can be considered the "father of Computer Science" (to the extent any single person can be thus credited), given that computing machines such as the <a href="http://en.wikipedia.org/wiki/Antikythera_mechanism">Antikythera Mechanism</a> have existed for many centuries.<br />
<br />
However, this point overlooks the fact that there is a difference between a "computing machine" and a "computer".<br />
<br />
In its broadest sense, <i>anything</i> that provides a convenient abstraction for making inferences about the world is a "computing machine". Therefore, in that sense, we've had computing machines for over 35,000 years – that's the age of the earliest counting implements known, the <a href="http://www.neatorama.com/2008/01/25/the-wonderful-world-of-early-computing/">Lebombo and Ishango Bones</a>.<br />
<br />
However, what differentiates a "computer" (in the sense we've used the word since Turing) from a computing machine is its generality: while a computing machine is designed to work on a specific problem class (no matter how large that class is), a computer can perform <i>any conceivable computation, including the simulation of other computers and computing machines, both less and more sophisticated than the computer itself.</i> This is no meaningless distinction – it is in fact the root of the computing revolution, as relatively simple silicon processors can run software programs much more complex than themselves.<br />
<br />
The Antikythera is certainly a remarkable device, but it's by no means a computer; it's merely a computing machine. You cannot, for example, use the Antikythera to solve differential equations, and you certainly cannot use it to simulate another device, say an arithmetic calculator. Likewise, the most sophisticated calculators still fail the Turing threshold, as they cannot be used to simulate the operation of other calculators.<br />
<br />
For a very enlightening account of Turing's breakthrough paper "On Computable Numbers, with an Application to the Entscheidungsproblem", which introduces Turing machines and many of the theoretical foundations of modern computing, I recommend reading the excellent <a href="http://www.charlespetzold.com/AnnotatedTuring/">The Annotated Turing</a>. It provides a wealth of explanations without which I wouldn't ever hope to understand Turing's paper, besides providing important historical context to its development and relevance. A must-read for any serious computer professional.<br />
<div>
<br /></div>Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comtag:blogger.com,1999:blog-1218335813680538098.post-2216075915881745652011-08-31T17:19:00.000-07:002011-08-31T17:19:13.824-07:00In Defense of Java Over C++A while ago <a href="http://posterous.com/people/4SnLETpfaa2Z"> Tim Robertson</a> <a href="http://sudoscientist.com/it-never-made-sense-to-learn-java">elaborated on his opinion</a> that Java compares rather unfavorably to C++, to the point it's not worth learning. His arguments are well known to the seasoned Java professional: limited features, poorer performance, and a sense of missing out on new developments and becoming more of a legacy platform (the infamous "new COBOL" argument). <br />
<br />
I beg to disagree. I have worked - and by "work" I mean professional, 8-to-5 programming work - on both C++ and Java projects for years now. They're both very good languages, and when used together can realize great things (hello Android). That said, I do find quite a few advantages to Java over C++: <br />
<ul>
<li>Less cruft. This has become a little less true since 1.5 with "generics", enums and the like, but still Java remains much more concise a language than C++. Arguing that C++ can do everything Java does and then some completely misses this point - often Java is a better tool than C++ precisely <i>because</i> it does less, while also <i>requiring</i> less;</li>
<li>A cleaner OO implementation. This is a continuation of my previous point but really deserves its own mention. I learned OO in C++ before getting to know Java, and I was relieved to see how much more straightforward Java's model was;</li>
<li>A unified memory model. In C++ you have a dozen ways to manage memory - local variables, <code>malloc</code> / <code>free</code>, the myriad variants of <code>new</code> / <code>delete</code>, etc - while in Java you have a couple primitive numeric types and then everything is dynamically allocated with <code>new</code> and garbage collected;</li>
<li>No invalid pointers. In Java an object variable is either null or pointing to a valid object. Whereas in C++ - well, do you feel lucky, punk?</li>
<li>No need for header files. It may be just laziness, but I'm really bothered by having to define every functional symbol twice, decide whether to <code>#include</code> dependencies on the header or source file, and so on;</li>
<li>Better namespace management. Java's <code>package</code> / <code>import</code> mechanism is a no-brainer, while C++'s namespace feature is more complicated, verbose, and incomplete - besides <code>using</code> symbols you still have to <code>#include</code> headers (did I mention headers suck?);</li>
<li>A standard, cross-platform mechanism for dynamic linking built right into the language. Whereas in C++ building a dynamic library is a project in itself (and woe if the thing fails to load), in Java you get it for free, as far as manual labor is concerned;</li>
<li>A more complete standard library. Sure, C++ is kind of catching up now - only that "now" actually means "a good ten years later".</li>
</ul>
What all this adds up to, is that even as C++ is in many respects a more powerful language than Java, programming in Java is a lot easier and quicker; things more often turn out right the first time around; and baffling, mysterious build and/or runtime issues are less frequent. Even when things do go haywire, the tight memory model and pervasive exception mechanism guarantee that you'll at least have a good idea as to where the problem manifested - compare this to C++ error handling, which often leaves programmers staring at a "segmentation fault" error message as though we were supposed to know from that what went wrong. <br />
<br />
Because of that, Java is more productive for daily programming, which makes it the preferred tool whenever it is up to the task. Perhaps it's not so much so if you're in academia or NASA, but for contractors implementing custom systems, such ease of development is a pretty big deal: most contracts have far stricter schedule than performance constraints. Getting something without glaring bugs and with acceptable performance <i>now</i> is much more valuable than getting it perfect in two months time. <br />
<br />
As for the "new COBOL" argument: first off, COBOL was only ever good to write business rules and the odd input form. It survives today at the expense of financial institutions who've grown so fat and lazy (or ignorant of their own processes) that rather than modernize their operations, they prefer to pay big bucks for a COBOL programmer and tools to keep everything as it ever was. Java on the other hand is all around, from home appliances and handsets to big computer clusters. I myself have used it for pretty much everything, from web service servers to serial port controllers and the odd AI gimmick. So please don't hold your breath for Java's collapse (or better yet, <i>do</i> hold your breath, by all means!) - it's not going to tail off any time soon, and surely no sooner than C++. <br />
<br />
Haters gonna hate, but professionals still have to weight on the right tool for the job. Whether it's Java, C++ or something else (I personally I'm very fond of Python), it should be decided on the languages' relative strengths and the task at hand, and not on nerd bigotry.Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comtag:blogger.com,1999:blog-1218335813680538098.post-77705736464779564692011-03-22T09:13:00.000-07:002011-03-22T09:13:25.526-07:00How a Bash Script Can Know Where It IsA colleague of mine wrote a shell script that is called from various other scripts, from different locations. He wanted his script to be able to discover the directory at which it is stored, but didn't know how; of course, calling <a href="http://www.manpagez.com/man/1/pwd/">pwd</a> inside the script would yield the current working directory (by default, the directory from which the original script was called), not the script's directory.<br />
<br />
After a little googling, I found this solution by netizen <a href="http://hintsforums.macworld.com/archive/index.php/t-73839.html">apokalyptik</a>:<pre class="prettyprint">#!/bin/bash
LSOF=$(lsof -p $$ | grep -E "/"$(basename $0)"$")
MY_PATH=$(echo $LSOF | sed -r s/'^([^\/]+)\/'/'\/'/1 2>/dev/null)
MY_PID=$$
MY_ROOT=$(dirname $MY_PATH)
MY_NAME=$(basename $0)
echo -e "PATH\t$MY_PATH"
echo -e "FILE\t$MY_NAME"
echo -e "CWD \t$MY_ROOT"
echo -e "PID \t$MY_PID"
</pre>The trick lies in the use of the <a href="http://www.manpagez.com/man/8/lsof/">lsof</a> command, which returns a list of currently open files. From there, the script uses <a href="http://www.manpagez.com/man/1/grep/">grep</a> to locate the entry corresponding to the currently running script, <a href="http://www.manpagez.com/man/1/sed/">sed</a> to separate the full path from the entry, then <a href="http://www.manpagez.com/man/1/dirname/">basename & dirname</a> to split the base directory and file name.<br />
<br />
This solution could be further enhanced by adding the current user to the <code>grep</code> regular expression: as it is, it could return the path to a same-named file opened by another user – but unless you work on a multi-user environment, it shouldn't make much difference.Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comtag:blogger.com,1999:blog-1218335813680538098.post-87023283701096691382011-03-15T14:03:00.000-07:002011-03-15T14:04:15.381-07:00To C Something in Less Words...<a href="http://users.softlab.ece.ntua.gr/~ttsiod/index.html">Thanassis Tsiodras</a> makes a good case on <a href="http://users.softlab.ece.ntua.gr/~ttsiod/cpp.html">C++ for C programmers</a>. His main point – which I wholeheartedly share – is that C++ allows algorithms to be generalized in ways not easily available in C (if at all), whithout the performance hit found in most other alternatives.<br />
<br />
Futhermore, both C programs and programmers can benefit from a more smooth transition, since the languages are mostly compatible at source-level. While it is my experience that, for new projects, one should not try to mingle C and C++ – it's better to go full C++ from the start, lests one risks creating an ugly mess of conflicting memory management paradigms – for ongoing projects hindered by the limitations of the C language, moderate doses of C++ may well be the right medicine.Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comtag:blogger.com,1999:blog-1218335813680538098.post-11729550364809628432011-03-10T08:19:00.000-08:002011-03-10T08:19:37.882-08:00Law and Science at The Edge of Human<a href="http://www.brookings.edu">Brookings</a> has a <a href="http://www.brookings.edu/papers/2011/0309_personhood_boyle.aspx">very nice article</a> on the legal (but more inherently, cultural) challenges scientific progress is likely to bring in the coming years, as technologies such as Computational Intelligence and Genetic Engineering further blur the line between human and non-human.<br />
<br />
Too often we focus entirely on the physical hazards poised by emerging technologies – particularly, what kind of beings it might spawn, and what their disposition towards mankind it would be. However, the social impact of their very existence may be just as disrupting as anything these theoretical new lifeforms might be able to do.<br />
<br />
In the typical Hollywood scenario, artificial beings – say, genetically enhanced mice – escape laboratory confinement and turn against humanity. However, in the real world, were a lab rat to display anything remotely resembling human-level cognition, it wouldn't even need try to escape: animal rights activists would be at the gates faster than you can spell "Sea Shepherd".Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comtag:blogger.com,1999:blog-1218335813680538098.post-37198334165955547912011-03-09T16:49:00.000-08:002011-03-09T16:49:03.082-08:00Hacking configurable indent width into BeautifulSoupMany a time I have been saved by the HTML parsing prowess of <a href="http://www.crummy.com/software/BeautifulSoup/">BeautifulSoup</a>. When it comes to beautifying HTML dumps for easier analysis it is king; however I have always been bothered by its unorthodox choice of indentation width (just one space per level). So the other day I took the time to hack configurable indent width into the code for version 3.2.0.<br />
<br />
The complete modified file is <a href="https://sites.google.com/site/machineawakeningsvault/home/BeautifulSoup.py?attredirects=0&d=1">here</a>; so long you're using version 3.2.0, you can just paste it over the original file at your installation. For details on the changes see the diff below:<pre class="prettyprint">***************
*** 544 ****
--- 545,547 ----
+ # Adjustable indentation patch
+ self.indentWidth = parser.indentWidth
+
***************
*** 752 ****
! space = (' ' * (indentTag-1))
--- 755 ----
! space = (self.indentWidth * (indentTag-1))
***************
*** 813 ****
! s.append(" " * (indentLevel-1))
--- 816 ----
! s.append(self.indentWidth * (indentLevel-1))
***************
*** 1080,1082 ****
! def __init__(self, markup="", parseOnlyThese=None, fromEncoding=None,
! markupMassage=True, smartQuotesTo=XML_ENTITIES,
! convertEntities=None, selfClosingTags=None, isHTML=False):
--- 1083,1088 ----
! def __init__(
! self, markup="", parseOnlyThese=None, fromEncoding=None,
! markupMassage=True, smartQuotesTo=XML_ENTITIES,
! convertEntities=None, selfClosingTags=None, isHTML=False,
! indentWidth=' '
! ):
***************
*** 1111 ****
--- 1118,1121 ----
+
+ # Adjustable indentation patch
+ self.indentWidth = indentWidth
+
</pre>Now setting a custom indent width is as simple as doing:<pre class="prettyprint">soup = BeautifulSoup(html_string.encode('utf-8'), indentWidth=' ')
beautiful_html_string = soup.prettify()
</pre>Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comtag:blogger.com,1999:blog-1218335813680538098.post-6032935432664553992011-03-09T16:06:00.000-08:002011-03-09T16:06:56.250-08:00Making numpy ndarray's HashableI am doing some research work using <a href="http://www.scipy.org/">numpy and scipy</a>, and I have been amazed by how fast they run; the other day, though, I stumbled on what seemed a roadblock. I wanted to use these quite large (length 1024) arrays as keys for storing data in Python dictionaries, however numpy's ndarray objects are not <a href="http://docs.python.org/glossary.html#term-hashable">hashable</a>; converting them to tuples would take forever, besides eating up an unbelievably large amount of memory. Searching the web provided no satisfactory answers.<br />
<br />
After a while, though, I realized that it would be simple to implement a hashable wrapper for my array objects. So here you have it:<pre class="prettyprint">from hashlib import sha1
from numpy import all, array, uint8
class hashable(object):
r'''Hashable wrapper for ndarray objects.
Instances of ndarray are not hashable, meaning they cannot be added to
sets, nor used as keys in dictionaries. This is by design - ndarray
objects are mutable, and therefore cannot reliably implement the
__hash__() method.
The hashable class allows a way around this limitation. It implements
the required methods for hashable objects in terms of an encapsulated
ndarray object. This can be either a copied instance (which is safer)
or the original object (which requires the user to be careful enough
not to modify it).
'''
def __init__(self, wrapped, tight=False):
r'''Creates a new hashable object encapsulating an ndarray.
wrapped
The wrapped ndarray.
tight
Optional. If True, a copy of the input ndaray is created.
Defaults to False.
'''
self.__tight = tight
self.__wrapped = array(wrapped) if tight else wrapped
self.__hash = int(sha1(wrapped.view(uint8)).hexdigest(), 16)
def __eq__(self, other):
return all(self.__wrapped == other.__wrapped)
def __hash__(self):
return self.__hash
def unwrap(self):
r'''Returns the encapsulated ndarray.
If the wrapper is "tight", a copy of the encapsulated ndarray is
returned. Otherwise, the encapsulated ndarray itself is returned.
'''
if self.__tight:
return array(self.__wrapped)
return self.__wrapped
</pre>Using the wrapper class is simple enough:<pre class="prettyprint">>>> from numpy import arange
>>> a = arange(0, 1024)
>>> d = {}
>>> d[a] = 'foo'
TypeError: unhashable type: 'numpy.ndarray'
>>> b = hashable(a)
>>> d[b] = 'bar'
>>> d[b]
'bar'
</pre>In my profiling sessions, adding the wrapped-up 1024-long arrays as keys to a dictionary amounted to no more overhead than adding the naked arrays to a list.Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comtag:blogger.com,1999:blog-1218335813680538098.post-21316054184126273892011-01-19T17:01:00.000-08:002011-01-21T03:03:13.598-08:00A Python API for Lego MINDSTORMS NXT Alpha Rex Design<b>Update:</b> <i>a properly-commented version of the code below made it to the <a href="http://nxt-python.googlecode.com/svn/trunk/examples/alpharex.py">nxt-python repository</a>. Yay me! :P</i><br /><br />Last week a friend of mine asked me to take his <a href="http://en.wikipedia.org/wiki/Lego_Mindstorms_NXT">Lego MINDSTORMS NXT</a> kit, build the <a href="http://www.active-robots.com/products/mindstorms4schools/building-instructions.shtml">Alpha Rex</a> design, and then demonstrate it to his father, who was a few days from returning to his home in Japan. Needless to say, having a chance to once again play with Lego – and in particular with the mind-blowingly nerdy, robotics-enabled NXT – was hardly unpleasant. In fact, while I was building the iconic bipedal robot – carefully and patiently following the instructions, taking the time to visualize how the pieces would fit together before actually connecting them, building the major parts before assembling them into the final model – I recognized just how similar to playing with Lego programming is, and how, during all those years of playtime, my mind was shaped for my current career in computing.<br /><br />So it's no surprise that when it came to putting Alpha Rex to work, I snubbed NXT's <a href="http://en.wikipedia.org/wiki/Lego_Mindstorms_NXT#NXT-G">Windows-based GUI tools</a>, and looked instead for some way to program it from my Linux netbook, preferably in Python. Sure enough, after a little Googling, I stumbled at <a href="http://code.google.com/p/nxt-python/">nxt-python</a>, "a Python driver/interface for the Lego Mindstorms NXT robot". Though at first I was a little discouraged by the motor interface (I was expecting something more in line with NXT-G's, where you can send commands to up to all three motors simultaneously), eventually I was able to find my way around the API, and even come to admire its simplicity and straightforwardness.<br /><br />The best part about nxt-python is Python itself. In just a couple hours I was able to write a rather sophisticated, object-oriented API to drive Alpha Rex and sample its various sensors – all in under 100 lines of code:<pre class="prettyprint"># alpharex.py<br /><br />from time import sleep<br /><br />from nxt.brick import Brick<br />from nxt.locator import find_one_brick<br />from nxt.motor import Motor, PORT_A, PORT_B, PORT_C<br />from nxt.sensor import Light, Sound, Touch, Ultrasonic<br />from nxt.sensor import PORT_1, PORT_2, PORT_3, PORT_4<br /><br /><br />FORTH = 100<br />BACK = -100<br /><br /><br />class AlphaRex(object):<br /> def __init__(self, brick='NXT'):<br /> if isinstance(brick, basestring):<br /> brick = find_one_brick(name=brick)<br /> <br /> self.brick = brick<br /> self.arms = Motor(brick, PORT_A)<br /> self.legs = [Motor(brick, PORT_B), Motor(brick, PORT_C)]<br /><br /> self.touch = Touch(brick, PORT_1)<br /> self.sound = Sound(brick, PORT_2)<br /> self.light = Light(brick, PORT_3)<br /> self.ultrasonic = Ultrasonic(brick, PORT_4)<br /><br /> def echolocate(self):<br /> return self.ultrasonic.get_sample()<br /> <br /> def feel(self):<br /> return self.touch.get_sample()<br /><br /> def hear(self):<br /> return self.sound.get_sample()<br /><br /> def say(self, line, times=1):<br /> for i in range(0, times):<br /> self.brick.play_sound_file(False, line + '.rso')<br /> sleep(1)<br /><br /> def see(self):<br /> return self.light.get_sample()<br /><br /> def walk(self, secs, power=FORTH):<br /> for motor in self.legs:<br /> motor.run(power=power)<br /> <br /> sleep(secs)<br /><br /> for motor in self.legs:<br /> motor.idle()<br /><br /> def wave(self, secs, power=100):<br /> self.arms.run(power=power)<br /> sleep(secs)<br /> self.arms.idle()<br /><br /><br />def wave_and_talk():<br /> robot = AlphaRex()<br /> robot.wave(1)<br /> robot.say('Object')<br /><br /><br />def walk_forth_and_back():<br /> robot = AlphaRex()<br /> robot.walk(10, FORTH)<br /> robot.walk(10, BACK)<br /><br /><br />def walk_to_object():<br /> robot = AlphaRex()<br /> while robot.echolocate() > 10:<br /> robot.walk(1, FORTH)<br /><br /> robot.say('Object.rso', 3)<br /><br /><br />if __name__ == '__main__':<br /> wave_and_talk()<br /></pre>Playing with Alpha Rex was real fun, but now I'm afraid my next move (since my friend allowed me to keep the kit a while longer) will be to dismantle it. Unable to turn around or move its ultrasound sensors, it's rather limited in its navigation capabilities – and I'm dying to check out whether I can build a <a href="http://en.wikipedia.org/wiki/Simultaneous_localization_and_mapping">SLAM</a>-capable robot out of the basic kit. At any rate, it'll certainly be pleasant to try.Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comtag:blogger.com,1999:blog-1218335813680538098.post-42373135262050243292010-11-04T05:12:00.000-07:002011-01-19T16:15:25.759-08:00Finding a Laser Dot in An Image with OpenCVEarlier today a friend of mine, who is working on an <a href="http://opencv.willowgarage.com/wiki/">OpenCV</a> project, asked me for a good way to locate a laser pointer in an image of a blank wall. I showed him two options:<br /><div><ol><li>The function <span class="Apple-style-span" style="font-family: monospace; font-size: 13px; white-space: pre; "><a href="http://opencv.willowgarage.com/documentation/c/object_detection.html#matchtemplate">cvMatchTemplate()</a></span> generates a likelyhood map for the position of a template within a larger image. See <a href="http://dasl.mem.drexel.edu/~noahKuntz/openCVTut6.html#Step 2">this tutorial</a> for a sample program;</li><li>Alternatively, since the searched object (the laser dot) distinguishes itself from the background by a primary color's (Red) concentration, it may be simpler to just isolate the image's Red channel with <span class="Apple-style-span" style="font-family: monospace; font-size: 13px; white-space: pre; "><a href="http://opencv.willowgarage.com/documentation/c/operations_on_arrays.html#setimagecoi">cvSetImageCOI()</a></span>, then use <span class="Apple-style-span" style="font-family: monospace; font-size: 13px; white-space: pre; "><a href="http://opencv.willowgarage.com/documentation/c/operations_on_arrays.html#minmaxloc">cvMinMaxLoc()</a></span> to return the position of the biggest-valued pixel:</li></ol><pre class="prettyprint">/*<br />Returns the most likely position of a laser dot within the input image. This is<br />defined as the position of the pixel with the biggest value in the Red<br />channel.<br /><br />Arguments:<br /><br />image<br /> Pointer to the input image, assumed 32-bit RGB.<br /><br />dot<br /> Pointer to the CvPoint instance to receive the output location.<br />*/<br />void cvSearchLaserDot(IplImage* image, CvPoint* dot) {<br /> double minVal, maxVal;<br /><br /> // Select's the image's first channel. Assuming this is an RGB image, the<br /> // Red channel will be selected.<br /> cvSetImageCOI(image, 1);<br /><br /> // Looks for the biggest-valued pixel on the above selected channel.<br /> cvMinMaxLoc(image, &minVal, &maxVal, NULL, dot, NULL);<br /><br /> // Resets the channel selection performed in the previous call to<br /> // cvSetImageROI().<br /> cvSetImageCOI(image, 0);<br />}<br /></pre><div>I did think of a third option – fitting a 2D gaussian function to the Red channel, then taking the gaussian's center as the location of the dot – but that would be a bit of an overkill, wouldn't it?</div></div>Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comtag:blogger.com,1999:blog-1218335813680538098.post-42727323449222249142010-10-26T03:52:00.000-07:002010-10-26T04:10:05.913-07:00Never Found A Programming Language I Couldn't Love<span class="Apple-style-span"><div><span class="Apple-style-span"><b><i><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: small; "><span class="Apple-style-span" >Notice:</span></span></span></i></b><i><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: small;"><span class="Apple-style-span" > this was originally <a href="http://www.codeproject.com/Lounge.aspx?fid=1159&select=3645250&tid=3645250">posted to the CodeProject's Lounge</a> in 2010-10-25, and featured in the Daily News the next day. Yay me! :P</span></span></span></i></span></div><div><span class="Apple-style-span"><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: small;"><span class="Apple-style-span" ><br /></span></span></span></span></div><span class="Apple-style-span"><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: small; "><span class="Apple-style-span" >Some say that "when the lady can't dance, she complains the band can't play". Generally that's my feeling towards complaints that such-and-such language is "bad", or that it produces "bad" code.<br /><br />I do agree that some languages lend themselves more than others to certain good (or bad) practices, and that some languages are better/worse suited than others for some problems. However, in my experience the most crippling problems are the result of poor programming style: the respective programmer community either doesn't know, or seems to forget how to write clear code once they join in.<br /><br />As way of example, this is my list of good practices for several widely-belittled languages. It's mostly about how you can use each language's specific features to uphold the principles of structured / OO programming:<br /><br /><b>Visual Basic</b><br /></span></span><ul><li><span class="Apple-style-span" style="font-size: small; "><span class="Apple-style-span" >Always use <code style="color: rgb(153, 0, 0); font: normal normal normal 11pt/normal 'Courier New', Courier, mono; ">Option Explicit</code> for enforcing variable declaration;</span></span></li><li><span class="Apple-style-span" style="font-size: small; "><span class="Apple-style-span" >Shun <code style="color: rgb(153, 0, 0); font: normal normal normal 11pt/normal 'Courier New', Courier, mono; ">Variant</code> variables – always use definite types;</span></span></li><li><span class="Apple-style-span" style="font-size: small;"><span class="Apple-style-span" >Horribly misnamed as they are, embrace "Classes" as the way to go for behavior encapsulation;</span></span></li><li><span class="Apple-style-span" style="font-size: small;"><span class="Apple-style-span" >Use the <code style="color: rgb(153, 0, 0); font: normal normal normal 11pt/normal 'Courier New', Courier, mono; "><span class="code-SDKkeyword" style="color: rgb(51, 153, 153); ">Object</span></code> type sparingly, but learn to recognize where it can be leveraged for generalizing algorithms;</span></span></li><li><span class="Apple-style-span" style="font-size: small;"><span class="Apple-style-span" >Well employed, the <code style="color: rgb(153, 0, 0); font: normal normal normal 11pt/normal 'Courier New', Courier, mono; ">On Error</code> machinery can make do as an effective Exception system;</span></span></li><li><span class="Apple-style-span" style="font-size: small;"><span class="Apple-style-span" >Prefer Collections over Arrays, and learn to explore their associative features.</span></span></li></ul><span class="Apple-style-span" style="font-size: small; "><span class="Apple-style-span" ><b>C</b><br /></span></span><ul><li><span class="Apple-style-span" style="font-size: small;"><span class="Apple-style-span" >Just because the language doesn't enforce a type-oriented form of programming doesn't mean you shouldn't. Always think about problems in terms of data types and accompanying operation (function) sets;</span></span></li><li><span class="Apple-style-span" style="font-size: small;"><span class="Apple-style-span" >Structure your code as a collection of header / source pairs, where the header defines (ideally) one data type and its API, and the source contains the implementations;</span></span></li><li><span class="Apple-style-span" style="font-size: small;"><span class="Apple-style-span" >Create more complex types by composing simpler ones, and likewise implement their API's in terms of the lower types' interfaces;</span></span></li><li><span class="Apple-style-span" style="font-size: small;"><span class="Apple-style-span" >When designing a type's API, remember to provide functions for dynamic instantiation, initialization of stack-defined instances, and deletion. Consider providing macros for access to struct fields, instead of conceding to direct dereference;</span></span></li><li><span class="Apple-style-span" style="font-size: small;"><span class="Apple-style-span" >Preserve the global namespace by defining symbols as <code style="color: rgb(153, 0, 0); font: normal normal normal 11pt/normal 'Courier New', Courier, mono; "><span class="code-keyword" style="color: blue; ">static</span></code> whenever they don't have to be seen outside their compilation units;</span></span></li><li><span class="Apple-style-span" style="font-size: small;"><span class="Apple-style-span" >Use function pointers for generalizing complex algorithms;</span></span></li><li><span class="Apple-style-span" style="font-size: small;"><span class="Apple-style-span" >Though dreadful when used carelessly, macros have great potential for simplifying programming interfaces – know when and how to use them.</span></span></li></ul><span class="Apple-style-span" style="font-size: small; "><span class="Apple-style-span" ><b>C++</b><br /></span></span><ul><li><span class="Apple-style-span" style="font-size: small;"><span class="Apple-style-span" >Forget <code style="color: rgb(153, 0, 0); font: normal normal normal 11pt/normal 'Courier New', Courier, mono; "><span class="code-keyword" style="color: blue; ">struct</span></code>'s, always use classes;</span></span></li><li><span class="Apple-style-span" style="font-size: small;"><span class="Apple-style-span" >Forget memory buffers and pointer arithmetic, use the STL collections instead;</span></span></li><li><span class="Apple-style-span" style="font-size: small;"><span class="Apple-style-span" >Prefer inline functions to macros;</span></span></li><li><span class="Apple-style-span" style="font-size: small;"><span class="Apple-style-span" >Avoid using pointers whenever possible. Use references for function arguments, inline definitions for class members, STL collections for dynamic instantiation, and locally-defined variables for automatic memory management;</span></span></li><li><span class="Apple-style-span" style="font-size: small;"><span class="Apple-style-span" >Leverage variable scope control for automating common operations, as logging function calls;</span></span></li><li><span class="Apple-style-span" style="font-size: small;"><span class="Apple-style-span" >C++ exception support is poor, but it's better than nothing – learn to take advantage of it;</span></span></li><li><span class="Apple-style-span" style="font-size: small;"><span class="Apple-style-span" >In general terms, do not stand with one foot on C and the other on C++ – jump in with both feet.</span></span></li></ul><b><span class="Apple-style-span" style="font-size: small;"><span class="Apple-style-span" >Scripting Languages (Perl, Javascript, PHP, etc)</span></span></b><span class="Apple-style-span" style="font-size: small; "><span class="Apple-style-span" ><br /></span></span><ul><li><span class="Apple-style-span" style="font-size: small;"><span class="Apple-style-span" >Just because you're writing to a script language doesn't mean you should forget how to program – structure your code around data types and/or classes, concise functions and specialist modules, just as you'd do on a "proper" systems programming language;</span></span></li><li><span class="Apple-style-span" style="font-size: small;"><span class="Apple-style-span" >Do not mingle your code to HTML. Write everything on separate modules; where inline code can't be avoided, provide as concise an interface as possible, ideally a single function call.</span></span></li></ul><span class="Apple-style-span" style="font-size: small; "><span class="Apple-style-span" >What about you? Do you think there are inherently "bad" languages, or is it essentially a matter of poor practice?</span></span></span></span></span>Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comtag:blogger.com,1999:blog-1218335813680538098.post-32922499145959040322010-10-19T05:24:00.000-07:002010-10-19T05:36:54.272-07:00WAC On Track to Deliver<div>The <a href="http://www.wholesaleappcommunity.com/default.aspx">Wholesale Applications Community</a> (WAC) is the heir to <a href="http://bondi.omtp.org/whatisbondi/WHAT%20WE%20DO/Home.aspx">BONDI</a> and the <a href="http://www.jil.org">Joint Innovation Lab</a> (JIL), which in turn were mobile application platforms based on extended versions of W3C's <a href="http://www.w3.org/TR/widgets-reqs/">Web Widgets</a> standard. The idea is enabling the use of web technologies (HTML, CSS and Javascript) to write full-blown mobile apps, as well as small homescreen-hanging "widgets". This is achieved by providing a web client runtime environment with an extended Javascript API, allowing access to underlying handset functions such as soft keys, camera, messaging system etc – though in my experience you can go a long way without ever crossing the limits of standard web programming.</div><div><br /></div><div>Ever since<a href="http://www.theregister.co.uk/2010/05/05/wacaday/"> its inception early this year</a> there has been quite reasonable doubts on WAC's ability to deliver. Following Android's avalanching success and a surge of me-too mobile platforms – <a href="http://www.theregister.co.uk/2010/05/07/bada_sdk/">Samsung's BADA</a>, <a href="http://www.theregister.co.uk/2010/05/10/hp_webos_tablet_rumor/">HP's plans for webOS</a>, Microsoft's <a href="http://www.theregister.co.uk/2010/03/16/microsoft_windows_phone_7_details/">Windows Phone 7</a> and the unnamed (and apparently stillborn) <a href="http://www.nttdocomo.com/pr/2010/001473.html">DOCOMO-led platform</a> were all announced around the same time – the whole enterprise seemed somewhat reactive. Add the mobile network operators' (WAC's principal sponsors) historical inability to agree on anything and poor innovation record, and it looked highly unlikely that WAC-enabled handsets would ever see the light of day, let alone take on already established smartphone platforms.</div><div><br /></div><div>It seems, however, that the rumors of WAC's unavoidable doom were highly exaggerated. Not only has the organization found <a href="http://www.theregister.co.uk/2010/10/18/wac_grows/">consistent support all across the mobile industry</a>, it has recently launched <a href="http://www.wholesaleappcommunity.com/About-Wac/DOCUMENTS/WAC1_0_High_Level_Technical_Specifications_18_Oct_2010.pdf">version 1.0 of its runtime specification</a> (though I feel this was rather rushed, as just two weeks ago it was still <a href="http://public.wholesaleappcommunity.com/redmine/embedded/wac2pubrev/index.html">requesting comments for an alleged "first draft"</a>). Since both BONDI and JIL managed to get shipped into actual feature phones before their somewhat forceful merger, I'd say there's now a definite chance we'll see WAC-enabled phones in the wild sooner or later.</div><div><br /></div><div>And I say the sooner the better. I have recently spent a couple weeks working on a proof-of-concept mobile widget for the BONDI platform, and I loved it. As it's mostly web programming with a couple extra Javascript objects, there is virtually no learning curve; it's also very portable, as whatever extensions you use can be abstracted into a Javascript file and swapped out when you want to run it on a desktop browser (which I actually did, as a shortcut for testing during development). Deployment was quite troublesome, as I had to upload the packaged application to the handset manufacturer's web portal, sign it, and then download it back; but overall the experience was very positive.</div><div><br /></div><div>Despite all the hype around smartphones, feature phones still dominate the market. Currently JME and Flash Light rule its app development roost; however, with the right promotion the WAC could very well topple them in no time, appealing as it would to the current generation of web-savvy developers with a platform they not only already know how to program, but for which they could effortlessly port already existing applications. If only the WAC associates can take it to the next step and provide enabled devices and developer support in a timely fashion, this might just be the beginning of the open, cross-vendor, cross-platform mobile application ecosystem we've been waiting for ever since the first days of WAP.</div>Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.comtag:blogger.com,1999:blog-1218335813680538098.post-21008300751625918762010-10-18T08:27:00.000-07:002010-10-18T09:02:04.517-07:00Real-world Functional Programming<div>There is a <a href="http://tratt.net/laurie/tech_articles/articles/why_dont_we_use_functional_programming_languages_more">recurrent discussion</a> in programming circles about the state-of-practice of functional programming languages, and the (presumably undeserved) little attention they get. It's curious to me that such arguments are usually centered on so-called "pure functional" languages such as Lisp, Scheme or Haskell (which isn't that "pure", but anyway), while totally ignoring how functional programming features have been consistently pouring into the mainstream programming landscape for more than a decade, through comparatively conservative efforts such as Javascript and Python.</div><div><br /></div><div>As a matter of fact, I do functional programming in Python a lot; only not right away. Often, when working on a problem, I'll start with a simple imperative solution based on top-level functions. Then, as the domain creeps ever larger and passing around values becomes awkward, I factor in a basic object model with only essential behavior moved to methods, while the bulk of processing remains on the functions.</div><div><br /></div><div>Finally, as recurring algorithms emerge, I start to refactor them into general implementations, customizable through object or function arguments. Eventually, most of my code is collected into highly general "portfolio" functions, while the "real" functions are mostly constructed by mixing and matching those. As a side-effect, closures and lambda functions abound.</div><div><br /></div><div>This approach doesn't have much of the obsessive recursivity found in classical functional programming (although I've at times marveled at the mind-blowing length of the function invocation chains it produces), but it does rely on the first-citizenship status Python grants to functions – in fact it wouldn't work otherwise. It isn't "pure" functional, as it isn't "pure" OO either; but it enables widespread code reuse, naturally leans toward cohesion & loose coupling, and is highly productive.</div><div><br /></div><div>Purity is a fool's errand. Just as it's not practical to have a "pure" object-oriented language (even Java has allowed for something akin to top-level function with static methods and imports), successful functional languages will have to compromise in order to accommodate its design features and the pragmatic needs of daily coding.</div>Heliohttp://www.blogger.com/profile/13823135713010326611noreply@blogger.com