Tag Archives: information technology

Stopping command-line applications programatically with Ctrl-C event from .Net – a working demo

Posted on by

The story behind

The problem of starting and stopping console/command-line applications from a host program is much more widespread, than one would think at the first glance. Starting an application and even redirecting its output is relatively easy to achieve. Stopping it nicely, on the other hand, is hard. “Nicely” is the keyword here. One can always call Kill() member function of a Process object. Yet that would terminate the child process immediately, without giving it a chance to do any clean-up, which in some cases can be disastrous.

On a Unix environment, a programmer would typically issue a SIGINT and/or SIGTERM signal, before resorting to SIGKILL. Windows has something almost, but not entirely, like this. It can generate a ConsoleControlEvent, which translates from a Ctrl-C or Ctrl-Break key press. These events are sent not to an application, but to the console/terminal with witch an application is registered and then propagated to all handlers in each application until one of the applications acknowledges the event. So, in reality, one sends a termination signal to all application, which are started from/with a particular console, and not just to a specific application. This, too, can spell trouble.

A popular work-around, which almost never works is to locate the window handle of a process (this assumes that the command line process has a window, which is not always the case) and sending a series of messages, to it, simulating key-down and key-up events, mapping to the ‘Ctrl’ and to the ‘C’ keys on the keyboard.

I was aiming for this approach, until I stumbled across a more elegant, and seemingly more fool-proof solution. Each solution will be outlined and briefly explained in the list below. Download and study the source code for the actual implementation (all worker functions are concentrated in a static class in Experiments.cs). If you happen to have a working solution for the PostMessage scenario, please share it with the community though the comments in this post.

Source code: [download]

Outline of the scenarios

If the started application (ping.exe) fails to stop (mainly as a result of your own experiments or of Scenario 2), hit the yellow Panic button to forcefully terminate it.
Only two scenarios represent real solutions to the problem – 1 and 4, with #4 being preferred. Scenario #2 was never completed due to time constraints (feel free to fill in the blanks ;)), while Scenario #3 is there for the sake of completeness.

  1. Start with a visible window using .Net, hide with pinvoke, run for 6 seconds, show with pinvoke, stop with .Net.

    This method is a mix of .Net and pinvoke. It stops the process by closing its command window, similar to clicking the ‘X’ button in the upper right corner. This generates a Ctrl-C event to all programs, registered with that console. To find the window, it must exist and be visible. Here I use method FindWindowHandleFromProcessObjectWithVisibleWindow() in the Experiments class, which uses .Net functionality. Alternatively I could have used a pinvoke method, which makes use of EnumWindows. This is implemented in FindWindowHandleFromPid() in the Experiments class.
    The downside of this method is that it briefly shows the commend window when starting and when stopping the process. Another problem is that a process takes long to close, it will be terminated. This method works fine for ping, but for my needs it was terminating the process to quickly.

  2. Pinvoke only: Start with invisible window, get window handle, run for 6 seconds, stop using window handle. (Unfinished!)

    It turned out that it was impossible to start a command window with an invisible, but still existing window using pinvoke CreateProcess. The start sequence ended up being the same as for Scenario 1, but bypassing .Net. Stopping a process with PostMessage did not work out either, even though I read of successful attempts. I abandoned this scenario as I did not want to use an inordinate amount of time on what started to look like a dead end.

  3. Brutal .Net: Start without window; run for 6 seconds; stop by killing the process.

    This scenario is is here for completeness. This is the most brutal method, which relies entirely on .Net functionality. When stopping the process, it simply kills it, without giving it any chance to clean up. Note that ping command does not display any summary when it is stopped in this fashion.

  4. Start without a window using .Net, run for 6 seconds, stop by attaching console and issuing ConsoleCtrlEvent.

    After finding a hint at the bottom of this thread, I finally bit the bullet and started the process without a window. When the time comes to stop a process, the parent attaches its console to the child, stops itself listening to Ctrl-C event and issues the event to the console, so that the attached child terminates. It then reinstates the Ctrl-C handler for itself and frees the console.

    UPDATE: Please read comment by Michal Svoboda below for an improved version for this scenario.

Resources

During the research phase, I got ideas and inspiration from the following posts and sites. Some them were dead ends, while others lead to something usable. The post in particular (the first link) was a real revelation, and is incorporated in the fourth scenario in the demo application.

Crypto Miners in Tray – Slim Universal Cryptocurrency Mining Front-End

Posted on by

The Story

After using GuiMiner for a couple of months, I got tired of a few of its shortcomings, so I designed and wrote a program, which meets my needs exactly and can do the following:

  • Allow for unattended 100% utilisation of mining resources
  • Allow mining selection between Bitcoin and various altcoins, depending on current profitability
  • Specify priority level for the individual miners
  • Specify processor affinity (even on Vista, where GuiMiner fails to do so)
  • Specify the backup pools
  • Can stop the miners nicely, by issuing Crl-C event, so that the miners have a chance to clean up. (GuiMiner simply terminated the miners, which resulted in NVidia display driver crash.)
  • Can watch the system for a set of conditions and modify miner state accordingly
  • Minimalistic GUI design
  • Be fully-configurable through a single XML file
  • Can run any miner, independent of its parameter signature, including rpcminer-4way.exe
  • Can run a set of programs prior to launching the miners – ideal for applying overclocking, using ClockTweak, or starting mining-proxy when when using getwork miner against Stratum pools.
  • Can keep the miners alive by checking their output and run state and re-starting them if they become unresponsive
  • Can auto-launch all the miners and hide itself in a tray with minimum fuss
  • Can be completely managed from the tray

Compatibility

I do not distribute any miners with this front-end. Indeed, it can support not only Bitcoin/Cryptocurrency miners, but other long running command-line applications, such as Folding@Home client. This applicatin has been tested with CGMiner, BFGMiner, Poclbm, Pooler’s CPUMiner, CudaMiner, RPC Miner, and Phoenix 2 Miner.

CGMiner must be started with --text-only or -T parameter to disable ncurses.

For CudaMiner and CpuMiner, remember to specify readSpeedFrom=”stderr” attribute in the config file.

NB! If you get a crash “Could not load file or assembly ‘System.Web.Extensions, Version=4.0.0.0’ when launching Crypto Miners in Tray, it means that you do not have Microsoft .NET Framework 4 installed. Download and install it from Microsoft site.

Download

Future versions:
– Add a custom column to the miner list. The text displayed in the column will be parsed from last miner output, using user-provided RegEx.
– Add tabbed view with one tab and one log file per miner.
– Add ready-to-use configuration examples.

Version 7.30: [Binary] [Source]
– Implement backup argument set rotation for KeepAlive function. This is handy for those miners that do not support specification of backup pools, such as cpuminer, poclbm and cudaminer.
– Fix an issue, where log file would be rotated twice when configuration file is reloaded.
– Fix a bug, where a miner in a profitability group would not get stopped if profitability changes while the miner is in a suspended by watcher state.

See the changelog for older versions and changes.

This program is represented at Bitcointalk software forum.

Donations

1DodoExzsNPvVRXFrgkKw6E259VjfUW8KhIf you find this program useful and use it on the daily basis, please support a good cause and donate to my Durrell Wildlife Conservation Trust bitcoin fund-raising drive

1DodoExzsNPvVRXFrgkKw6E259VjfUW8Kh

Configuration

An example configuration file is included with the binary. Edit it to reflect your setup, prerequisites and miners and rename it to BtMinersInTray.xml

Screenshot

Crypto Miners In Tray

Crypto Miners In Tray - Balloon Status Tooltip Crypto Miners In Tray - Context Menu

The GUI is largely self-explanatory. Running miners are shown in green, stopped miners – in orange and mis-configured miners (wrong workingDir and/or application info) – in red. One or more miners can be started or stopped by marking the checkbox by their names. The program starts directly into system tray. Double-click the icon to show the GUI window. The probram can be minimised back to tray with (_) or closed with (X). Reloading config will stop all the running miners, read the new configuration and start the miners with any updates.

Almost all operations can be performed without opening the GUI window, through the use of the context menu and the status balloon tooltip.

Changelog and archived versions

Version 7.21: [Binary] [Source]
– Fixed a bug, introduced in 7.20, where “Start with Windows” would write incorrect application executable name to registry.
– Fixed logging to file so that the text is auto-flushed, allowing to view the log file while it is being written to.
– Refactored registry accessing methods to facilitate testing.
– Keep using old/stale data from CoinChoose if the site’s API temporarily stops supplying data for a coin.
– Improved error handling in log to file code.

Version 7.20: (retracted)
– Added Execute action to watchers. External command can be launched once watcher condition either becomes positive, negative or in both cases.
– Added possibility to continuously append the output log to file. Log files are rotated with a timestamp on log window clearing. Logging to file is disable by default.

Version 7.10: [Binary] [Source]
– Made it possible to define global variables in the configuration file. These variables can then be referenced in any node or attribute, making it easier to update multiple places at once.
– Fixed a bug, where the initial state of “Run with Windows” was always checked contrary to the actual state of affairs.

Version 7.00: [Binary] [Source]
– Implemented “miner profitability grouping”, where the program would choose one of the miners in a group, targeting the same card/hardware for the most profitable coin, using the information from CoinChoose.com. At most one miner in a group can run at any given time. A group is treated as one miner: Hitting “Start” on any miner in a group would start the most profitable miner. Hitting “Stop” on any miner in a group would stop the running miner in that group.
– GUI changes, moving all buttons to the toolbar.
– GUI change, allowing resize of the program window and of the relative proportion between miner list and output text box.
– GUI window size and splitter position between miner list and the log window are saved in the registry and restored upon next program launch.
– Fixed a bug, where custom-selected config file would not be loaded when the program is launched with Windows.
– The program’s name has been changed to “Crypto Miners in Tray” to denote that miners targeting different cryptocoins can be managed from the application. The executable’s name remains the same for backward compatibility.

Version 6.01: [Binary] [Source]
– Fixed a bug where keepAlive with threshold would stop working if a miner’s hashing speed would drop to 0 right after a successful keepAlive check.

Version 6.00: [Binary] [Source]
– Complete rewrite of the Process launching code, encapsulating the .Net Process class, so as to be able to peek on the stdout/stderr output pipe from the miners, solving the problem with some miners (pooler’s cpuminer) not showing any output until they exit. (Thanks to WebMaka for a lead.)
– All speeds in the balloon tooltip are now shown in the “dot” decimal notation, culture independent
– Added Speed column to the GUI list of miners.
– Added milliseconds to the log timestamp.
– Made config file path in the status bar into a clickable link, which would open Explorer at that path.
– Added restartEvery attribute to the miner node, allowing miners to be force restarted at certain hour intervals.
– Added readSpeedFrom attribute, allowing the user where the hashing speed is parsed from: stdout or stderr.
– Possibility to specify how the miner reports its speed in the list and the tooltip – K, M or G – through the new displaySpeedIn attribute. The total in the tooltip and GUI is always in M.
– Fixed a bug in hashrate-based keepAlive threshold calculation, which resulted in frequent false positives.
– Fixed a problem with disabled auto-scroll. Windows interop EM_GETSCROLLPOS returns scaled down 16-bit values even though it has a 32-bit Point structure at its disposal, resulting in erronous behaviour when the size of the RichTextBox content height exceeds 65535 pixels. A work-around has been applied, which alliviated the problem, though the text would not stay completely still.
– Fixed an off-by-one error in average speed calculation.
– The donation hint now points to Durrell Wildlife Conservation Trust fund-raising: 1DodoExzsNPvVRXFrgkKw6E259VjfUW8Kh

Version 5.10: [Binary] [Source]
– Added “Run with Windows” checkbox to the GUI and tray context menu.
– Improved positional handling of the output text box when autoScroll feature is disabled.
– KeepAlive Hits column now shows counters for each category.
– Reset KeepAlive Hits counters when a miner is manually restarted.
– Added threshold attribute to the checkHashRate keepAlive option. An average hashing speed during the keepAlive check interval, which is below this threshold will trigger miner restart.
– Added a small and unobtrusive donation hint to the main GUI screen. 😉

Version 5.00: [Binary] [Source]
– Long-running prerequisites are now stopped when the program exits or the config file is reloaded.
– Prerequisites can now have keepAlive setting, watching for Process and Output conditions.
– Replaced custom-drawn checkbox-containing ListBox with ListView control. It is now multi-selectable, and without checkboxes.
– ListView box now contains detailed information about each miner’s configuration, including current priority and keepAlive hit count. Ideally it should stay on 0. If the value is growing, it means that you either have a misconfigured keepAlive condition, so that the function kicks in more than it should, or that you have overclocked your GPU too much and it keeps crashing the driver.
– KeepAlive now reports reasons for restarting the miner (in the log window).
– The application now targets .Net Framework ver. 4.0 and is built with explicit platform target set to x86.

Version 4.10: [Binary] [Source]
– Fixed a bug, where setting keepAlive to 0 would cause a crash
– keepAlive is back to being a child node of a miner, and can now be configured with interval and restart conditions: Process, Output, and HashRate.
– Fixed a rare occasion, when both the user and a watcher starting a miner at exactly the same time would result in two instances of the miner to be launched.
– Fixed handling of start/stop/restart watcher events to make it more resilient.
– Added killDelay miner attribute, so it is possible to tweak the time between Ctrl-C event is sent to the miner and the miner is deemed unresponsive to it and is forcefully killed.
– waitForExit attribute of a prerequisite execute statement is renamed to delay and can now specify a delay in seconds, in addition to special values of ‘no’ and ‘forever’.

Version 4.00: [Binary] [Source]
– Added locks around critical sections in miner start and stop routines. Addition of asynchronous watchers introduced a possibility of race conditions.
– Changed the colour of miner entries in the list and context menu: Green – running, Orange – suspended by watcher, Red – stopped. Misconfigured/invalid miners are no longer shown in the lists.
– Changed the way watchers are defined: condition and action parameters are now specified in the body of the watcher element. See example config file.
– Watcher check interval can now be configured on a per-miner level.
– Added two new watcher actions: RestartWithArguments and ChangeAffinity.
– Priority action is renamed into ChangePriority.
– Added three new watcher conditions: BatteryPower, TimeInterval and DaysOfWeek.
– Renamed Enabled miner setting to autoStart to better reflect what that setting actually does.
– Made averageWindow setting miner-specific.
– Application watcher can now specify multiple processes to watch.
– Tray icon is now visible while GUI is maximized (unless Stealth Mode TrayIcon is specified).
– Seconds are now displayed in the log timestamp.
– Breaking change to the config format: id, autoStart, keepAlive, averageWindow, and watcherCheckInterval are now attributes of the <watch> node, instead of being child nodes. This is done so as to make the config file less cluttered
– Breaking change to the config format: waitForExit is now an attributes of the <execute> node, instead of being a child node.

Version 3.00: [Binary] [Source]
– Introduced watchers that can observe such conditions as human activity or a running application and start/stop/change priority of the miners accordingly.
– Stealth config option which can either disable on-hover balloon tooltip or hide the tray icon altogether, in which case showing GUI is done by running the program again.
– Keep-alive setting is moved to individual miners (because of pooler’s cpumine’s faulty output mechanism). The default is now ‘off’.
– Total hashing speed info moved to status bar.
– Added possibility to specify any config file through the command line argument.
– Added possibility to open any config file through the GUI.
– Information about currently loaded config file is in the status bar.
– AutoScroll can be togged from GUI.
– Stealth mode and average window info is in the status bar.
– Fixed a bug, where user-stopped miner would sometimes be treated as dead and restarted.
– Fixed a bug, where the program would crash if no averageWindow was specified in the config (missing default).
– Improved stdout parsing heuristics to read correct speed info from CGMiner output and to avoid false positives.
– Brought most of the business logic under test, reordered some code, latest additions programmed using TDD.
– Various bug fixes as the result of unit testing.

Version 2.20: [Binary] [Source]
– Individual and total hashing speed is now shown in on-hover balloon tooltip in tray icon.
– Tray icon now has a context menu with the commands to open the GUI, stop all miners, reload config, start/stop individual miners, and exit.
– Average window for hashing speed calculation is now configurable.
– Checkboxes of the misconfigured miners are now disabled.

Version 2.10: [Binary] [Source]
– Fixed a rare bug, where an exception would be thrown if checkbox list redraw occurred at the same time as config file reload.
– Simplified routine that stops the miners, eliminating the need for briefly-flashing command line windows when starting miners.
– Added colour tagging of the output window: Green – program output; Blue – miner stdout, Red – miner stderr.
– Made output textbox read-only.
– Average hashing speed across all miners is now displayed in the GUI.

Version 2.01: [Binary] [Source]
– Fixed hiding of the icon in task switcher, when the application is minimised to tray.

Version 2.00: [Binary] [Source]
– Initial stable release

The first version was published at BitcoinTalk newbie forum, but was still very much a work in progress.

File locator web front-end in PHP

Posted on by

When my file server resided in the Windows environment, I made use of the Everything search engine to index the files and to search for them both locally and through Everything’s built-in web server.

This latter functionality is what I wanted to replicate once I built the ZFS-based FreeBSD file server and moved to it. All UNIX flavours have the locate command, which will use a pre-built database to quickly find a string in file names and paths on your server. So, the obvious solution was to install Apache and PHP and write a web front-end for locate.

An alternative option is to use Solr, a Lucene search engine front-end. I, however, wished to have something simpler and custom-made. This was also a good opportunity to explore a new programming language.

I’ve never written PHP code before, as my main area is ASP.NET and C#, but learning the ropes of PHP was an enjoyable task and it is always good to learn another language. The result can be seen below.

The program will search for an arbitrary string in the locate database, optionally ignoring the case. Another option lets the user to restrict the search to the last segment of the path, thus avoiding flooding with nearly duplicate hits if the string is located only in the directory portion of the path. The program will also highlight the hits.

This is how the simple UI of the program looks like:

Update 1

I’ve added some more desired functionality:

  • the program can now update the underlying locate database through the web interface
  • it now accepts ‘*’ and ‘?’ wildcards in the search string and highlights the results appropriately
  • it can now give direct links to the located content
  • it can now search for string containing Unicode charachters
  • highlighting is made Lynx-friendly

Forcing database update involves running the update script as root, which will then su as user nobody. Apache (httpd) runs under a limited user www (or suchlike). To overcome this obstacle, I used a solution, suggested in this Stack Overflow thread:

  1. Modify update.launcher.c (code below) to point to the update script, which is typically located in /etc/periodic/weekly/310.locate
  2. #gcc update.launcher.c -o update.launcher
  3. #chown root update.launcher
  4. #chmod u=rwx,go=xr,+s update.launcher
  5. Place the program on your server and modify UPDATE_SCRIPT_LAUNCHER constant in the program
  6. Verify that LOCATE_DB_FILE constant points to the database file, so that the porgram is able to report the state of the database

Remember to change the value in SEARCH_ROOT constant, which limits the search location range.

If you want the program to display direct links to the located content, perform the following 2 steps:

  1. Create a symbolic link to the root of your searchable content, as defined in SEARCH_ROOT
  2. Update VIEW_SYMLINK_PREFIX constant to point to that symlink, relative to web root or relative to the locator.html placement. (If this constant is not defined, the program will not generate any links.)

There are a few caveats and assumptions:

  • There is no thorough error checking involved
  • Unicode search is always case sensitive

locator.html

Download







File Locator









Search for: 
 
(wildcards * and ? are allowed)

 /> 
Ignore case

 /> 
Search in last segment only

 















File name database is currently being updated.
Search results may be inaccurate.

';
    }

    $ret = array();
    $command = 'locate ' . ($ignoreCase ? '-i "' : '"') . 
		SEARCH_ROOT . '*' . $searchString . '*"';

    exec($command, $ret);

    $word = str_replace(array("?", "*"), array(".", ".+"), $searchString);

    foreach ($ret as $line)
    {
	if($lastSegmentSearch && !foundInLastSegment($line, $word, $ignoreCase))
	{
	    continue;
	}
	
	$find = highlight($line, $word, $ignoreCase);
	if(defined("VIEW_SYMLINK_PREFIX"))
	{
	    $link = str_replace(SEARCH_STRING, VIEW_SYMLINK_PREFIX, $line);
	    print '[View] ';
	}
	print "$find
\n";
    }
}

function showDatabaseState()
{
    if(updateLocatorIsRunning())
    {
	print 'File name database is currently being updated.';
	return;
    }
    else
    {
	clearstatcache();
	date_default_timezone_set('UTC');
	$dbtime = date("D, d.m.Y, H:i:s", filemtime(LOCATE_DB_FILE));
	print 'File name database was last updated on ' .
		$dbtime . '';
    }
}

function updateDatabase()
{
    if(updateLocatorIsRunning())
    {
	print 'File name database is already being updated!';
	return;
    }
    $command = UPDATE_SCRIPT_LAUNCHER . " > /dev/null 2>&1 &";
    exec($command);
    sleep(1);

    if(updateLocatorIsRunning())
    {
	print 'Started updating file name database.';
    }
    else
    {
	print 'File name database updator failed to start.';
    }
}

function updateLocatorIsRunning()
{
    $ret = array();
    $command = "ps -U nobody -o command";
    exec($command, $ret);
    foreach ($ret as $line)
    {
	if(strstr($line, "locate.updatedb"))
	{
	    return true;
	}
    }
    return false;
}

function foundInLastSegment($line, $searchString, $ignoreCase)
{
    $search = '/(?=[^\/]+$)' . $searchString . ($ignoreCase ? '/i' : '/');
    return preg_match($search, $line);
}

function highlight($text, $word, $ignoreCase)
{
    return preg_replace("/($word)/U" . ($ignoreCase ? "i" : ""),
                        "$1",
                         $text);
}
?>

update.launcher.c

Download

#include 
#include 
#include 

int main (int argc, char *argv[])
{
    setuid (0);
    system ("/bin/sh /etc/periodic/daily/320.locate");
    return 0;
}

RegEx to match a substring after a delimiter

Posted on by

They say that if you have a problem and want to use RegEx to solve it, then you have two problems. So true! 🙂

My specific problem was that I wanted to search for a string within a substring after a delimiter sign, more precisely, in the last segment of a path. Here is an example:

/some/test_path/to/search/with_a_Test_file.txt

The RegEx, searching without case sensitivity for “test” should return a match only for the portion of the string after the last “/”.
All suggestions, which I could find on StackOverflow, concerned with matching the entire file name and not a portion of it, so I had to learn some advanced RegEx. Fast.

The answer was something, called “lookahead”, which is well explained at Regular-Expressions.info site.

The resulting RegExt string looks like some serious swearing in a cartoon bubble… 🙂 Here is the code, which is accepted by PHP’s preg_match() function:

/(?=[^\/]+$)test/i

According to my (rather limited) understanding of RegEx, the first portion in the parenthesis, after the “?=”is the lookahead, which matched the entire file name after the last “/”, then comes the search substring, “test”, which operates on that result and, finally, “/i” is the switch, instructing a case-insensitive match.

Adding disks by label in ZFS and making those labels stick around

Posted on by

When I stared building my new file server, I decided to add the disks to ZFS vdevs by label and not by the device id, i.e:

#glabel label l1 /dev/ada0
#glabel label l2 /dev/ada1

After a reboot, those labelled disks suddenly started to show up as /dev/ada0 and /dev/ada1 again and the labels disappeared from /dev/label directory.

For the existing disks, I tried to offline each disk in turn and re-label it. A new problem turned up then: I could not replace the /dev/adaX offlined disks with the same labelled ones, as zpool gave an error of the device “is part of active pool”.

After some further searching, I found out that I had to zero out the first and the last megabyte of the disk before labelling it and replacing in zpool:

#dd if=/dev/zero of=/dev/ada0 bs=1m count=1
#dmesg | grep ada0
<read the block count value, subtract 2048 and provide the result to the seek switch below>
#dd if=/dev/zero of=/dev/ada0 seek=358746954
#glabel label l1 /dev/ada0
#zpool replace zstore /dev/ada0 label/l1

At this point zpool status was again showing labels. However, after the next reboot, the labels were gone again and I was pretty frustrated. Back to the search engine.

On page 3 of some discussion of this matter, I noticed two additional steps, which should fix the problem. After performing the steps above and re-labelling and re-placing the disks, I issued:

#zpool export zstore
#zpool import -d /dev/label zstore

The -d switch is what instructs zpool to read the disk references from a specific directory and it makes the labels stick around.

When I added subsequent new disks to the pool, I followed these steps to make the labels stick and to avoid re-labelling at a later point:

  1. Zero-out the first and the last part of each disk that will comprise the new vdev (especially important if the disk has been in use before and does not come staight from the factory)
  2. Label each disk with glabel
  3. #zpool add zstore raidz label/l5 label/l6 etc….
  4. #zpool export zstore
  5. #zpool import -d /dev/label zstore

And the labels never disappeared again.

This same procedure can be applied to labelling your ZIL and LARC devices.

Reporting correct space usage for Samba shared ZFS volumes

Posted on by

ZFS is all the rage now and there are lots of tutorials and how-to’s out there covering most of the topics. There is one issue, for which I could not find any ready solution. When sharing a zfs volume over Samba, Windows would report incorrect total volume size. More precisely, Windows would always show the same size for both total size and free size and both values will be changing as the volume gets used.
This is obviously not what we want. Some digging uncovered that Samba relies internally on the result of the df program, which will report incorrect values for ZFS systems. More digging lead to this page and to the man pages of smb.conf, showing that it is possible to override space usage detection behaviour by creating a custom script and pointing Samba server to it using the following entry in smb.conf:

[global]
dfree command = /usr/local/bin/dfree

The following bash script is where the magic lies (tested on FreeBSD):

#!/bin/sh

CUR_PATH=`pwd`

let USED=`zfs get -o value -Hp used $CUR_PATH` / 1024 > /dev/null
let AVAIL=`zfs get -o value -Hp available $CUR_PATH` / 1024 > /dev/null

let TOTAL = $USED + $AVAIL > /dev/null

echo $TOTAL $AVAIL

And the following is a variation, which works on Linux (courtesy commenter nem):

#!/bin/bash

CUR_PATH=`pwd`

USED=$((`zfs get -o value -Hp used $CUR_PATH` / 1024)) > /dev/null
AVAIL=$((`zfs get -o value -Hp available $CUR_PATH` / 1024)) > /dev/null

TOTAL=$(($USED+$AVAIL)) > /dev/null

echo $TOTAL $AVAIL

Make sure to check the comments section, as several variations of this script are posted there, for example taking account for both ZFS and non-ZFS shares on the same system!

I can’t use zpool list as it reports the total size for the pool, including parity disks, so the total size might be greater than the real usable total size.
zfs list could have been used if there was a way to display the information in bytes and not in human-readable form of varying granularity.
The solution was to use zfs get and then normalise the values reported to Samba to the 1024 byte blocks. (I tried providing the third, optional, parameter of 1 as mentioned in the man pages, but Samba seemed to have trouble parsing really large byte values, so I ended up doing the normalisation in the script).

Also, I can’t rely on the $1 input parameter to the script, as it turned out to always be equal to ‘.’, which is usable for df, but not for zfs. This ‘.’ lead me to check the working directory of the invocation and, bingo, it turned out to be the root path of the requested volume, so I could simply get the value from pwd and pass it to zfs.

Laptop Whine Killer

Posted on by

Background: Lenovo’s W510 laptops produce two kinds of high-pitched noises. One of those is especially prominent and irritating. It appears mainly when the laptop is running on battery power in a near-idle state.

Solution: Searching on the Internet produced this 2-year old thread on Lenovo’s support forums.
The only working solution to this embarrassing whine was to start an audio playback, so that the CPU would not enter a deep power-saving state. The other alternative was to disable power management in BIOS. Disabling power management is not desirable as it negatively affects battery consumption, while silent sound playback has almost no negative impact.
I put together a small system tray program, which does just that – it will play an embedded silent 1-second long WAV file in an indefinite loop. This will effectively stop the whine at the cost of a slightly shorter battery life. The effective CPU usage while running the suppression is less than 0.1%.
The program can automatically sense if your laptop runs on AC power and disable the whine suppression loop. This feature is not enabled by default as there is still some whine present when on AC power. It is possible to adjust the settings from the system tray context menu.

[Download]
[Source code]

Version history:
1.00 – 03.07.2012 – Initial release.
1.01 – 04.07.2012 – Save automatic detection setting state to the registry. Improve power state detection reliability.
1.02 – 04.07.2012 – Allow only one instance of running application.
1.10 – 25.04.2013 – Added an option to auto-launch the program together with Windows.

The icon used in this program comes from Icon Archive.

LGA 2011 CPU socket backplate cooling modification

Posted on by

I’ve recently set about building a new powerful machine for myself, with a firm intent to overclock it for 24/7 use. So, while being powerful, it would also need to run reasonably cool. The ingredients were:

The copper plate of the cooling system was lapped to complete flatness, using wet sandpapering with corn size of 320, 600 and finally 1000. Everything was mounted with a minute amount of Arctic Silver 5 so as not to degrade thermal conductivity. I also set up a push-pull fan system around the radiator to more efficiently dispose of the excess hot air.

After the thermal compound went through its curing period and the core and socket temperatures stabilised, I started overclocking the system, while running Prime95 for load and System Information for Windows (SIW) for monitoring. SIW gives more information, than CoreTemp. More specifically, you get to see the package and socket temperatures.

I hit the heat limit for 24/7 usage at 3.8GHz. The cores were running at about 73C, while the socket temperature kept at 64C with the ambient room temperature of 24C. And here comes the interesting part…

The socket backplate is well exposed in the Carbide 400R case, thanks to the big window in the motherboard mounting plate.

The backplate was impossibly hot to the touch – 64C is not to be taken lightly. I then decided to try something, which I think (according to a few Google searches) haven’t been done before – to attach a cooling system to the backplate.

Corsair Carbide 400R has a nice feature: two characteristic bulges on either side of the case. On the left-hand side, that would allow you to put an oversized heat sink on your CPU. On the right-hand side it leaves quite a lot of empty space between the motherboard surface and the case wall.

I had two old heat sinks, which served in its time on a Pentium III machine. I also had a low profile fan, though I don’t remember whence it came from. After some measuring (important!), sawing, lapping and screw fastening, I had the following construction:

This construction got permanently glued to the central part of the backplate using Arctic Alumina Thermal Adhesive. (That’s why measuring beforehand is vitally important – the keyword here is “permanently”).

Finally I cut two appropriately-sized bits from a larger heat sink, lapped them as well and attached them to the upper part of the backplate, so that they partially overhang the motherboard. The upper part is where the most heat was generated, so adding those two heat sink bits was intended to even out the odds. The final working assembly looks like this:

I’ve configured the fan to pull the air from the heat sink. even though I have high air pressure within the case and dust filters on all 4 intake fans, there is bound to be some minute dust particles left in the air stream. By pulling the air from the motherboard, I ensure some air circulation around the heat sinks, while avoiding bombarding the back of the motherboard with the aforementioned dust particles. The fan is also connected through two Noctua speed reducers, so the fan is not audible, while still pulling a significant amount of air.

So, did all this hassle pay off? Oh, yes. The socket temperature went down by 14 degrees to 50C and the average core temperature went down by 6 degrees to 67.

I have ultimately overclocked the CPU to 4.3GHz (VCore 1.310V) and under the heaviest load of Prime95, the socket now warms to 57C with the core averaging 71C. With everyday loads (like running SMP build of Folding@Home) the temperatures stay within 55C / 68C.

If you decide to do anything like this with your system, make sure that you do thorough measurements and that the heat sink (with an optional fan) will fit under the case cover. And remember that the adhesive is PERMANENT. 😉

UPDATE: I have newly gone all-out and added passive cooling to the power regulator backplate, which was also running too hot. This required some sawing and lapping of the heatsink set, sold by Akasa. I also changed the fan over the CPU backplate as the old one started making noises (those small fans don’t last long – maximum a couple of months). That’s how it looks now:

Power backplate coolingPower backplate cooling

PROTECT IP / SOPA Breaks the Internet

Posted on by


This video speaks for itself!
Don’t let 2012 be like 1981!

Tell Congress not to censor the internet NOW! – http://www.fightforthefuture.org/pipa

PROTECT-IP is a bill that has been introduced in the Senate and the House and is moving quickly through Congress. It gives the government and corporations the ability to censor the net, in the name of protecting “creativity”. The law would let the government or corporations censor entire sites– they just have to convince a judge that the site is “dedicated to copyright infringement.” The government has already wrongly shut down sites without any recourse to the site owner. Under this bill, sharing a video with anything copyrighted in it, or what sites like Youtube and Twitter do, would be considered illegal behavior according to this bill. According to the CBO, this bill would cost us $47 million tax dollars a year — that’s for a fix that won’t work, disrupts the internet, stifles innovation, shuts out diverse voices, and censors the internet. This bill is bad for creativity and does not protect your rights.