Tag Archives: programming

Stopping command-line applications programatically with Ctrl-C event from .Net – a working demo

Posted on by

The story behind

The problem of starting and stopping console/command-line applications from a host program is much more widespread, than one would think at the first glance. Starting an application and even redirecting its output is relatively easy to achieve. Stopping it nicely, on the other hand, is hard. “Nicely” is the keyword here. One can always call Kill() member function of a Process object. Yet that would terminate the child process immediately, without giving it a chance to do any clean-up, which in some cases can be disastrous.

On a Unix environment, a programmer would typically issue a SIGINT and/or SIGTERM signal, before resorting to SIGKILL. Windows has something almost, but not entirely, like this. It can generate a ConsoleControlEvent, which translates from a Ctrl-C or Ctrl-Break key press. These events are sent not to an application, but to the console/terminal with witch an application is registered and then propagated to all handlers in each application until one of the applications acknowledges the event. So, in reality, one sends a termination signal to all application, which are started from/with a particular console, and not just to a specific application. This, too, can spell trouble.

A popular work-around, which almost never works is to locate the window handle of a process (this assumes that the command line process has a window, which is not always the case) and sending a series of messages, to it, simulating key-down and key-up events, mapping to the ‘Ctrl’ and to the ‘C’ keys on the keyboard.

I was aiming for this approach, until I stumbled across a more elegant, and seemingly more fool-proof solution. Each solution will be outlined and briefly explained in the list below. Download and study the source code for the actual implementation (all worker functions are concentrated in a static class in Experiments.cs). If you happen to have a working solution for the PostMessage scenario, please share it with the community though the comments in this post.

Source code: [download]

Outline of the scenarios

If the started application (ping.exe) fails to stop (mainly as a result of your own experiments or of Scenario 2), hit the yellow Panic button to forcefully terminate it.
Only two scenarios represent real solutions to the problem – 1 and 4, with #4 being preferred. Scenario #2 was never completed due to time constraints (feel free to fill in the blanks ;)), while Scenario #3 is there for the sake of completeness.

  1. Start with a visible window using .Net, hide with pinvoke, run for 6 seconds, show with pinvoke, stop with .Net.

    This method is a mix of .Net and pinvoke. It stops the process by closing its command window, similar to clicking the ‘X’ button in the upper right corner. This generates a Ctrl-C event to all programs, registered with that console. To find the window, it must exist and be visible. Here I use method FindWindowHandleFromProcessObjectWithVisibleWindow() in the Experiments class, which uses .Net functionality. Alternatively I could have used a pinvoke method, which makes use of EnumWindows. This is implemented in FindWindowHandleFromPid() in the Experiments class.
    The downside of this method is that it briefly shows the commend window when starting and when stopping the process. Another problem is that a process takes long to close, it will be terminated. This method works fine for ping, but for my needs it was terminating the process to quickly.

  2. Pinvoke only: Start with invisible window, get window handle, run for 6 seconds, stop using window handle. (Unfinished!)

    It turned out that it was impossible to start a command window with an invisible, but still existing window using pinvoke CreateProcess. The start sequence ended up being the same as for Scenario 1, but bypassing .Net. Stopping a process with PostMessage did not work out either, even though I read of successful attempts. I abandoned this scenario as I did not want to use an inordinate amount of time on what started to look like a dead end.

  3. Brutal .Net: Start without window; run for 6 seconds; stop by killing the process.

    This scenario is is here for completeness. This is the most brutal method, which relies entirely on .Net functionality. When stopping the process, it simply kills it, without giving it any chance to clean up. Note that ping command does not display any summary when it is stopped in this fashion.

  4. Start without a window using .Net, run for 6 seconds, stop by attaching console and issuing ConsoleCtrlEvent.

    After finding a hint at the bottom of this thread, I finally bit the bullet and started the process without a window. When the time comes to stop a process, the parent attaches its console to the child, stops itself listening to Ctrl-C event and issues the event to the console, so that the attached child terminates. It then reinstates the Ctrl-C handler for itself and frees the console.

    UPDATE: Please read comment by Michal Svoboda below for an improved version for this scenario.

Resources

During the research phase, I got ideas and inspiration from the following posts and sites. Some them were dead ends, while others lead to something usable. The post in particular (the first link) was a real revelation, and is incorporated in the fourth scenario in the demo application.

Crypto Miners in Tray – Slim Universal Cryptocurrency Mining Front-End

Posted on by

The Story

After using GuiMiner for a couple of months, I got tired of a few of its shortcomings, so I designed and wrote a program, which meets my needs exactly and can do the following:

  • Allow for unattended 100% utilisation of mining resources
  • Allow mining selection between Bitcoin and various altcoins, depending on current profitability
  • Specify priority level for the individual miners
  • Specify processor affinity (even on Vista, where GuiMiner fails to do so)
  • Specify the backup pools
  • Can stop the miners nicely, by issuing Crl-C event, so that the miners have a chance to clean up. (GuiMiner simply terminated the miners, which resulted in NVidia display driver crash.)
  • Can watch the system for a set of conditions and modify miner state accordingly
  • Minimalistic GUI design
  • Be fully-configurable through a single XML file
  • Can run any miner, independent of its parameter signature, including rpcminer-4way.exe
  • Can run a set of programs prior to launching the miners – ideal for applying overclocking, using ClockTweak, or starting mining-proxy when when using getwork miner against Stratum pools.
  • Can keep the miners alive by checking their output and run state and re-starting them if they become unresponsive
  • Can auto-launch all the miners and hide itself in a tray with minimum fuss
  • Can be completely managed from the tray

Compatibility

I do not distribute any miners with this front-end. Indeed, it can support not only Bitcoin/Cryptocurrency miners, but other long running command-line applications, such as Folding@Home client. This applicatin has been tested with CGMiner, BFGMiner, Poclbm, Pooler’s CPUMiner, CudaMiner, RPC Miner, and Phoenix 2 Miner.

CGMiner must be started with --text-only or -T parameter to disable ncurses.

For CudaMiner and CpuMiner, remember to specify readSpeedFrom=”stderr” attribute in the config file.

NB! If you get a crash “Could not load file or assembly ‘System.Web.Extensions, Version=4.0.0.0’ when launching Crypto Miners in Tray, it means that you do not have Microsoft .NET Framework 4 installed. Download and install it from Microsoft site.

Download

Future versions:
– Add a custom column to the miner list. The text displayed in the column will be parsed from last miner output, using user-provided RegEx.
– Add tabbed view with one tab and one log file per miner.
– Add ready-to-use configuration examples.

Version 7.30: [Binary] [Source]
– Implement backup argument set rotation for KeepAlive function. This is handy for those miners that do not support specification of backup pools, such as cpuminer, poclbm and cudaminer.
– Fix an issue, where log file would be rotated twice when configuration file is reloaded.
– Fix a bug, where a miner in a profitability group would not get stopped if profitability changes while the miner is in a suspended by watcher state.

See the changelog for older versions and changes.

This program is represented at Bitcointalk software forum.

Donations

1DodoExzsNPvVRXFrgkKw6E259VjfUW8KhIf you find this program useful and use it on the daily basis, please support a good cause and donate to my Durrell Wildlife Conservation Trust bitcoin fund-raising drive

1DodoExzsNPvVRXFrgkKw6E259VjfUW8Kh

Configuration

An example configuration file is included with the binary. Edit it to reflect your setup, prerequisites and miners and rename it to BtMinersInTray.xml

Screenshot

Crypto Miners In Tray

Crypto Miners In Tray - Balloon Status Tooltip Crypto Miners In Tray - Context Menu

The GUI is largely self-explanatory. Running miners are shown in green, stopped miners – in orange and mis-configured miners (wrong workingDir and/or application info) – in red. One or more miners can be started or stopped by marking the checkbox by their names. The program starts directly into system tray. Double-click the icon to show the GUI window. The probram can be minimised back to tray with (_) or closed with (X). Reloading config will stop all the running miners, read the new configuration and start the miners with any updates.

Almost all operations can be performed without opening the GUI window, through the use of the context menu and the status balloon tooltip.

Changelog and archived versions

Version 7.21: [Binary] [Source]
– Fixed a bug, introduced in 7.20, where “Start with Windows” would write incorrect application executable name to registry.
– Fixed logging to file so that the text is auto-flushed, allowing to view the log file while it is being written to.
– Refactored registry accessing methods to facilitate testing.
– Keep using old/stale data from CoinChoose if the site’s API temporarily stops supplying data for a coin.
– Improved error handling in log to file code.

Version 7.20: (retracted)
– Added Execute action to watchers. External command can be launched once watcher condition either becomes positive, negative or in both cases.
– Added possibility to continuously append the output log to file. Log files are rotated with a timestamp on log window clearing. Logging to file is disable by default.

Version 7.10: [Binary] [Source]
– Made it possible to define global variables in the configuration file. These variables can then be referenced in any node or attribute, making it easier to update multiple places at once.
– Fixed a bug, where the initial state of “Run with Windows” was always checked contrary to the actual state of affairs.

Version 7.00: [Binary] [Source]
– Implemented “miner profitability grouping”, where the program would choose one of the miners in a group, targeting the same card/hardware for the most profitable coin, using the information from CoinChoose.com. At most one miner in a group can run at any given time. A group is treated as one miner: Hitting “Start” on any miner in a group would start the most profitable miner. Hitting “Stop” on any miner in a group would stop the running miner in that group.
– GUI changes, moving all buttons to the toolbar.
– GUI change, allowing resize of the program window and of the relative proportion between miner list and output text box.
– GUI window size and splitter position between miner list and the log window are saved in the registry and restored upon next program launch.
– Fixed a bug, where custom-selected config file would not be loaded when the program is launched with Windows.
– The program’s name has been changed to “Crypto Miners in Tray” to denote that miners targeting different cryptocoins can be managed from the application. The executable’s name remains the same for backward compatibility.

Version 6.01: [Binary] [Source]
– Fixed a bug where keepAlive with threshold would stop working if a miner’s hashing speed would drop to 0 right after a successful keepAlive check.

Version 6.00: [Binary] [Source]
– Complete rewrite of the Process launching code, encapsulating the .Net Process class, so as to be able to peek on the stdout/stderr output pipe from the miners, solving the problem with some miners (pooler’s cpuminer) not showing any output until they exit. (Thanks to WebMaka for a lead.)
– All speeds in the balloon tooltip are now shown in the “dot” decimal notation, culture independent
– Added Speed column to the GUI list of miners.
– Added milliseconds to the log timestamp.
– Made config file path in the status bar into a clickable link, which would open Explorer at that path.
– Added restartEvery attribute to the miner node, allowing miners to be force restarted at certain hour intervals.
– Added readSpeedFrom attribute, allowing the user where the hashing speed is parsed from: stdout or stderr.
– Possibility to specify how the miner reports its speed in the list and the tooltip – K, M or G – through the new displaySpeedIn attribute. The total in the tooltip and GUI is always in M.
– Fixed a bug in hashrate-based keepAlive threshold calculation, which resulted in frequent false positives.
– Fixed a problem with disabled auto-scroll. Windows interop EM_GETSCROLLPOS returns scaled down 16-bit values even though it has a 32-bit Point structure at its disposal, resulting in erronous behaviour when the size of the RichTextBox content height exceeds 65535 pixels. A work-around has been applied, which alliviated the problem, though the text would not stay completely still.
– Fixed an off-by-one error in average speed calculation.
– The donation hint now points to Durrell Wildlife Conservation Trust fund-raising: 1DodoExzsNPvVRXFrgkKw6E259VjfUW8Kh

Version 5.10: [Binary] [Source]
– Added “Run with Windows” checkbox to the GUI and tray context menu.
– Improved positional handling of the output text box when autoScroll feature is disabled.
– KeepAlive Hits column now shows counters for each category.
– Reset KeepAlive Hits counters when a miner is manually restarted.
– Added threshold attribute to the checkHashRate keepAlive option. An average hashing speed during the keepAlive check interval, which is below this threshold will trigger miner restart.
– Added a small and unobtrusive donation hint to the main GUI screen. 😉

Version 5.00: [Binary] [Source]
– Long-running prerequisites are now stopped when the program exits or the config file is reloaded.
– Prerequisites can now have keepAlive setting, watching for Process and Output conditions.
– Replaced custom-drawn checkbox-containing ListBox with ListView control. It is now multi-selectable, and without checkboxes.
– ListView box now contains detailed information about each miner’s configuration, including current priority and keepAlive hit count. Ideally it should stay on 0. If the value is growing, it means that you either have a misconfigured keepAlive condition, so that the function kicks in more than it should, or that you have overclocked your GPU too much and it keeps crashing the driver.
– KeepAlive now reports reasons for restarting the miner (in the log window).
– The application now targets .Net Framework ver. 4.0 and is built with explicit platform target set to x86.

Version 4.10: [Binary] [Source]
– Fixed a bug, where setting keepAlive to 0 would cause a crash
– keepAlive is back to being a child node of a miner, and can now be configured with interval and restart conditions: Process, Output, and HashRate.
– Fixed a rare occasion, when both the user and a watcher starting a miner at exactly the same time would result in two instances of the miner to be launched.
– Fixed handling of start/stop/restart watcher events to make it more resilient.
– Added killDelay miner attribute, so it is possible to tweak the time between Ctrl-C event is sent to the miner and the miner is deemed unresponsive to it and is forcefully killed.
– waitForExit attribute of a prerequisite execute statement is renamed to delay and can now specify a delay in seconds, in addition to special values of ‘no’ and ‘forever’.

Version 4.00: [Binary] [Source]
– Added locks around critical sections in miner start and stop routines. Addition of asynchronous watchers introduced a possibility of race conditions.
– Changed the colour of miner entries in the list and context menu: Green – running, Orange – suspended by watcher, Red – stopped. Misconfigured/invalid miners are no longer shown in the lists.
– Changed the way watchers are defined: condition and action parameters are now specified in the body of the watcher element. See example config file.
– Watcher check interval can now be configured on a per-miner level.
– Added two new watcher actions: RestartWithArguments and ChangeAffinity.
– Priority action is renamed into ChangePriority.
– Added three new watcher conditions: BatteryPower, TimeInterval and DaysOfWeek.
– Renamed Enabled miner setting to autoStart to better reflect what that setting actually does.
– Made averageWindow setting miner-specific.
– Application watcher can now specify multiple processes to watch.
– Tray icon is now visible while GUI is maximized (unless Stealth Mode TrayIcon is specified).
– Seconds are now displayed in the log timestamp.
– Breaking change to the config format: id, autoStart, keepAlive, averageWindow, and watcherCheckInterval are now attributes of the <watch> node, instead of being child nodes. This is done so as to make the config file less cluttered
– Breaking change to the config format: waitForExit is now an attributes of the <execute> node, instead of being a child node.

Version 3.00: [Binary] [Source]
– Introduced watchers that can observe such conditions as human activity or a running application and start/stop/change priority of the miners accordingly.
– Stealth config option which can either disable on-hover balloon tooltip or hide the tray icon altogether, in which case showing GUI is done by running the program again.
– Keep-alive setting is moved to individual miners (because of pooler’s cpumine’s faulty output mechanism). The default is now ‘off’.
– Total hashing speed info moved to status bar.
– Added possibility to specify any config file through the command line argument.
– Added possibility to open any config file through the GUI.
– Information about currently loaded config file is in the status bar.
– AutoScroll can be togged from GUI.
– Stealth mode and average window info is in the status bar.
– Fixed a bug, where user-stopped miner would sometimes be treated as dead and restarted.
– Fixed a bug, where the program would crash if no averageWindow was specified in the config (missing default).
– Improved stdout parsing heuristics to read correct speed info from CGMiner output and to avoid false positives.
– Brought most of the business logic under test, reordered some code, latest additions programmed using TDD.
– Various bug fixes as the result of unit testing.

Version 2.20: [Binary] [Source]
– Individual and total hashing speed is now shown in on-hover balloon tooltip in tray icon.
– Tray icon now has a context menu with the commands to open the GUI, stop all miners, reload config, start/stop individual miners, and exit.
– Average window for hashing speed calculation is now configurable.
– Checkboxes of the misconfigured miners are now disabled.

Version 2.10: [Binary] [Source]
– Fixed a rare bug, where an exception would be thrown if checkbox list redraw occurred at the same time as config file reload.
– Simplified routine that stops the miners, eliminating the need for briefly-flashing command line windows when starting miners.
– Added colour tagging of the output window: Green – program output; Blue – miner stdout, Red – miner stderr.
– Made output textbox read-only.
– Average hashing speed across all miners is now displayed in the GUI.

Version 2.01: [Binary] [Source]
– Fixed hiding of the icon in task switcher, when the application is minimised to tray.

Version 2.00: [Binary] [Source]
– Initial stable release

The first version was published at BitcoinTalk newbie forum, but was still very much a work in progress.

File locator web front-end in PHP

Posted on by

When my file server resided in the Windows environment, I made use of the Everything search engine to index the files and to search for them both locally and through Everything’s built-in web server.

This latter functionality is what I wanted to replicate once I built the ZFS-based FreeBSD file server and moved to it. All UNIX flavours have the locate command, which will use a pre-built database to quickly find a string in file names and paths on your server. So, the obvious solution was to install Apache and PHP and write a web front-end for locate.

An alternative option is to use Solr, a Lucene search engine front-end. I, however, wished to have something simpler and custom-made. This was also a good opportunity to explore a new programming language.

I’ve never written PHP code before, as my main area is ASP.NET and C#, but learning the ropes of PHP was an enjoyable task and it is always good to learn another language. The result can be seen below.

The program will search for an arbitrary string in the locate database, optionally ignoring the case. Another option lets the user to restrict the search to the last segment of the path, thus avoiding flooding with nearly duplicate hits if the string is located only in the directory portion of the path. The program will also highlight the hits.

This is how the simple UI of the program looks like:

Update 1

I’ve added some more desired functionality:

  • the program can now update the underlying locate database through the web interface
  • it now accepts ‘*’ and ‘?’ wildcards in the search string and highlights the results appropriately
  • it can now give direct links to the located content
  • it can now search for string containing Unicode charachters
  • highlighting is made Lynx-friendly

Forcing database update involves running the update script as root, which will then su as user nobody. Apache (httpd) runs under a limited user www (or suchlike). To overcome this obstacle, I used a solution, suggested in this Stack Overflow thread:

  1. Modify update.launcher.c (code below) to point to the update script, which is typically located in /etc/periodic/weekly/310.locate
  2. #gcc update.launcher.c -o update.launcher
  3. #chown root update.launcher
  4. #chmod u=rwx,go=xr,+s update.launcher
  5. Place the program on your server and modify UPDATE_SCRIPT_LAUNCHER constant in the program
  6. Verify that LOCATE_DB_FILE constant points to the database file, so that the porgram is able to report the state of the database

Remember to change the value in SEARCH_ROOT constant, which limits the search location range.

If you want the program to display direct links to the located content, perform the following 2 steps:

  1. Create a symbolic link to the root of your searchable content, as defined in SEARCH_ROOT
  2. Update VIEW_SYMLINK_PREFIX constant to point to that symlink, relative to web root or relative to the locator.html placement. (If this constant is not defined, the program will not generate any links.)

There are a few caveats and assumptions:

  • There is no thorough error checking involved
  • Unicode search is always case sensitive

locator.html

Download







File Locator









Search for: 
 
(wildcards * and ? are allowed)

 /> 
Ignore case

 /> 
Search in last segment only

 















File name database is currently being updated.
Search results may be inaccurate.

';
    }

    $ret = array();
    $command = 'locate ' . ($ignoreCase ? '-i "' : '"') . 
		SEARCH_ROOT . '*' . $searchString . '*"';

    exec($command, $ret);

    $word = str_replace(array("?", "*"), array(".", ".+"), $searchString);

    foreach ($ret as $line)
    {
	if($lastSegmentSearch && !foundInLastSegment($line, $word, $ignoreCase))
	{
	    continue;
	}
	
	$find = highlight($line, $word, $ignoreCase);
	if(defined("VIEW_SYMLINK_PREFIX"))
	{
	    $link = str_replace(SEARCH_STRING, VIEW_SYMLINK_PREFIX, $line);
	    print '[View] ';
	}
	print "$find
\n";
    }
}

function showDatabaseState()
{
    if(updateLocatorIsRunning())
    {
	print 'File name database is currently being updated.';
	return;
    }
    else
    {
	clearstatcache();
	date_default_timezone_set('UTC');
	$dbtime = date("D, d.m.Y, H:i:s", filemtime(LOCATE_DB_FILE));
	print 'File name database was last updated on ' .
		$dbtime . '';
    }
}

function updateDatabase()
{
    if(updateLocatorIsRunning())
    {
	print 'File name database is already being updated!';
	return;
    }
    $command = UPDATE_SCRIPT_LAUNCHER . " > /dev/null 2>&1 &";
    exec($command);
    sleep(1);

    if(updateLocatorIsRunning())
    {
	print 'Started updating file name database.';
    }
    else
    {
	print 'File name database updator failed to start.';
    }
}

function updateLocatorIsRunning()
{
    $ret = array();
    $command = "ps -U nobody -o command";
    exec($command, $ret);
    foreach ($ret as $line)
    {
	if(strstr($line, "locate.updatedb"))
	{
	    return true;
	}
    }
    return false;
}

function foundInLastSegment($line, $searchString, $ignoreCase)
{
    $search = '/(?=[^\/]+$)' . $searchString . ($ignoreCase ? '/i' : '/');
    return preg_match($search, $line);
}

function highlight($text, $word, $ignoreCase)
{
    return preg_replace("/($word)/U" . ($ignoreCase ? "i" : ""),
                        "$1",
                         $text);
}
?>

update.launcher.c

Download

#include 
#include 
#include 

int main (int argc, char *argv[])
{
    setuid (0);
    system ("/bin/sh /etc/periodic/daily/320.locate");
    return 0;
}

RegEx to match a substring after a delimiter

Posted on by

They say that if you have a problem and want to use RegEx to solve it, then you have two problems. So true! 🙂

My specific problem was that I wanted to search for a string within a substring after a delimiter sign, more precisely, in the last segment of a path. Here is an example:

/some/test_path/to/search/with_a_Test_file.txt

The RegEx, searching without case sensitivity for “test” should return a match only for the portion of the string after the last “/”.
All suggestions, which I could find on StackOverflow, concerned with matching the entire file name and not a portion of it, so I had to learn some advanced RegEx. Fast.

The answer was something, called “lookahead”, which is well explained at Regular-Expressions.info site.

The resulting RegExt string looks like some serious swearing in a cartoon bubble… 🙂 Here is the code, which is accepted by PHP’s preg_match() function:

/(?=[^\/]+$)test/i

According to my (rather limited) understanding of RegEx, the first portion in the parenthesis, after the “?=”is the lookahead, which matched the entire file name after the last “/”, then comes the search substring, “test”, which operates on that result and, finally, “/i” is the switch, instructing a case-insensitive match.