Bitcoin Fundraising for Durrell Wildlife Conservation Trust

Featured

Posted on by
3


I have been a long-standing donor and adopter at Durrell Wildlife conservation Trust. I have newly updated my membership to lifetime and increased my adoption level to Silver (Mr S Sokolov – Norway). Still, there is much more that can and should be done for the conservation of wildlife on our fragile planet.

The famous naturalist and the founder of the first zoo (on Jersey), aimed at study and conservation of endangered species, Gerald Durrell once wrote: “Animals and plants have nobody to speak for them except us, the human beings who share the world with them but do not own it.”

When I was first introduced to Bitcoins, I thought that they would be an ideal medium for micro-donations, especially with the growing user- and miner base of the new global currency.

This is a fund-raising campaign that I would like to start, using Bitcoins as the base currency. Please make your micro-donations to the address below or scan the address QR code in the upper right corner of the post

1DodoExzsNPvVRXFrgkKw6E259VjfUW8Kh

If you don’t have Bitcoins, please donate or otherwise help Durrell Wildlife Conservation Trust in any way you can, by proceeding to their site.

At the end of each month, if at least 1 Bitcoin is raised, I will convert BTC to GBP and transfer them to the Trust as a Donation from my account there. I will then update this post stating how much was transferred to the Trust in terms of GBP.

For the hall of fame, you are welcome to leave comments, stating how much you donated, or embed messages in the blockchain.

Disclaimer: This fund-raising drive is my personal initiative and is not endorsed by Durrell Wildlife Conservation Trust.

This topic at Bitcointalk forum

A Little Bit of Everything

Featured

Posted on by
Reply

This site is a hub to my various projects and older stuff, that I did on the ‘net since early 1990′s.

My interests include photography, computer science, gaming, biking, and some video editing. Most of these things are (or will be) presented here. Some of the photos, which I create, are sold through various stock photo sites at very affordable prices. I invite you to visit them through the links, provided in the menu above. My latest additions to iStockPhoto collection can be seen at the bottom of the page.

I also host a rather popular (at least according to search results) HelpPC Reference Library with extensive information on x86 assembly and interrupts.

Please leave a comment in this post if you have anything to say about the site and its contents in general.

GMail new compose – simply terrible!

Posted on by
Reply

I have no words! It is so wrong on so many levels:

The size

Too small.

The placement

Lower right corner? Seriously? So you feel compelled to bend down to peek at it. My first intuitive motion was to try and grab the window bar an drag that pop-up to the upper portion of the screen, to my eye-level. But no, it would not budge!

Functionality

Lacking! If I wanted a chat program (which I don’t), I’d use one. E-mails are for serious communications.

Maximise

One more click is required to pop this tiny compose out into a…. new browser window?! I don’t want a new browser window! I use tabs!

In stead of conclusion

In a comment by user Chalchihuite here I read a brilliant would-be dialogue between a Google developer an interior decorator, which would demonstrate the thought process (or lack thereof) behind the new Compose mis-feature:

Decorator: …And then we’ll put the couch over here.

Gmail programmer: Oh, our data show that people only sit on living room couches 20% of the time. So it makes more sense to keep the couch in the garage, and tell people to carry it in whenever they feel like watching TV. By the way, we put the TV down in the basement. Look for the box labelled with a mysterious icon.

Decorator: Wouldn’t it be a lot easier to just keep the couch and TV here, so they’re available whenever people feel like using them?

Gmail programmer: Nah, people just have to get used to the new system. (Chuckles) Besides, what are they going to do, move?

Decorator: Well…okay. Was there anything that you wanted to put in place of the items you took out?

Gmail programmer: Ick, no. I want to shrink the room down to a 6′-by-2′-by-1′ box. Rigorous analysis shows that a majority of room users are under 6 feet in height, and less than 2 feet in width. And basic logic indicates that no one requires more space than the volume of one’s own body. It’s just good, efficient design.

Decorator: You realize you’re describing a coffin, right? Besides, what about people who want to sit down and watch TV? Or who are more than 6 feet tall?

Gmail programmer: Geez, what a whiner. Those weirdos can open the trapdoor marked with a mysterious icon and ride the chute down to a convenient subterranean grotto, where they’re free to indulge in perversions like “sitting.” It’ll be easy for them as long as they’re careful not to open the other trapdoor, marked with the other mysterious icon. That leads to a cobra pit.

Decorator: I know you believe that a tiny, bare box will be simpler and cleaner than a large, well-furnished room, but I think you’re actually creating a lot of needless complexity. You’re making it difficult and frustrating for people to do basic things, which they’re already used to doing without any problem at all.

Gmail programmer: DATA! EFFICIENCY! TABLETS! TEXTING! TWITTER! TEENAGERS! CLEAN! CLEAN! CLEEEEEAAAAANNNNN!!!!!!!!

Free Bitcoin sites with minimum fuss

Posted on by
1

Mining for bitcoins on GPUs is fun, especially if you have 3 video cards in one system. This income can, however, be supplemented by some foraging for free micro-bitcoins. This does not bring much to your wallet, but the total amount can be acceptable if the foraging does not require much of your time.

Below is a selection of sites, which I found to be more productive and less demanding on your attention. This list is compiled from a much wider selection, presented in this thread on Bitcointalk.

  1. BitVisitor – Visit a random site after captcha to get a delayed payout. Many sites can be visited with 5 minute intervals.
  2. CoinTube – Earn 10 uBTC for Every Video Watched.
  3. Free bitcoins.me – Every 30 minutes, delayed payout
  4. El Bitcoin Gratis – Every 30 minutes, delayed payout
  5. BTC4You – Every 30 minutes, delayed payout
  6. FaucetBTC – Every 30 minutes, delayed payout
  7. Can Has Bitcoin – Every 30 minutes, delayed payout
  8. Daily Bitcoin – Hourly, accumulated payout-on-demand
  9. CoinAd – Hourly with registration, accumulated payout-on-demand
  10. Bitcoins 4 Me! – Hourly, delayed payout
  11. BitHits.info – 3 hits per IP per day per address, delayed payout
  12. CoinReaper – Daily, has its own bonus payout, aggregator over several other sites
  13. Bitcoin Tree – Daily, accumulated payout-on-demand
  14. Bitcoiner.net – Daily, payout-on-demand
  15. Bunny Run – Daily, delayed payout
  16. Netlookup.se – Daily, delayed payout
  17. cointicket.org – Daily, delayed payout

If you find this collection useful, a small portion of your foraging is welcome as a donation to 1DodoExzsNPvVRXFrgkKw6E259VjfUW8Kh as a contribution to my Durrell Wildlife Conservation Trust bitcoin fund-raising drive.

Stopping command-line applications programatically with Ctrl-C event from .Net – a working demo

Posted on by
1

The story behind

The problem of starting and stopping console/command-line applications from a host program is much more widespread, than one would think at the first glance. Starting an application and even redirecting its output is relatively easy to achieve. Stopping it nicely, on the other hand, is hard. “Nicely” is the keyword here. One can always call Kill() member function of a Process object. Yet that would terminate the child process immediately, without giving it a chance to do any clean-up, which in some cases can be disastrous.

On a Unix environment, a programmer would typically issue a SIGINT and/or SIGTERM signal, before resorting to SIGKILL. Windows has something almost, but not entirely, like this. It can generate a ConsoleControlEvent, which translates from a Ctrl-C or Ctrl-Break key press. These events are sent not to an application, but to the console/terminal with witch an application is registered and then propagated to all handlers in each application until one of the applications acknowledges the event. So, in reality, one sends a termination signal to all application, which are started from/with a particular console, and not just to a specific application. This, too, can spell trouble.

A popular work-around, which almost never works is to locate the window handle of a process (this assumes that the command line process has a window, which is not always the case) and sending a series of messages, to it, simulating key-down and key-up events, mapping to the ‘Ctrl’ and to the ‘C’ keys on the keyboard.

I was aiming for this approach, until I stumbled across a more elegant, and seemingly more fool-proof solution. Each solution will be outlined and briefly explained in the list below. Download and study the source code for the actual implementation (all worker functions are concentrated in a static class in Experiments.cs). If you happen to have a working solution for the PostMessage scenario, please share it with the community though the comments in this post.

Source code: [download]

Outline of the scenarios

If the started application (ping.exe) fails to stop (mainly as a result of your own experiments or of Scenario 2), hit the yellow Panic button to forcefully terminate it.
Only two scenarios represent real solutions to the problem – 1 and 4, with #4 being preferred. Scenario #2 was never completed due to time constraints (feel free to fill in the blanks ;) ), while Scenario #3 is there for the sake of completeness.

  1. Start with a visible window using .Net, hide with pinvoke, run for 6 seconds, show with pinvoke, stop with .Net.

    This method is a mix of .Net and pinvoke. It stops the process by closing its command window, similar to clicking the ‘X’ button in the upper right corner. This generates a Ctrl-C event to all programs, registered with that console. To find the window, it must exist and be visible. Here I use method FindWindowHandleFromProcessObjectWithVisibleWindow() in the Experiments class, which uses .Net functionality. Alternatively I could have used a pinvoke method, which makes use of EnumWindows. This is implemented in FindWindowHandleFromPid() in the Experiments class.
    The downside of this method is that it briefly shows the commend window when starting and when stopping the process. Another problem is that a process takes long to close, it will be terminated. This method works fine for ping, but for my needs it was terminating the process to quickly.

  2. Pinvoke only: Start with invisible window, get window handle, run for 6 seconds, stop using window handle. (Unfinished!)

    It turned out that it was impossible to start a command window with an invisible, but still existing window using pinvoke CreateProcess. The start sequence ended up being the same as for Scenario 1, but bypassing .Net. Stopping a process with PostMessage did not work out either, even though I read of successful attempts. I abandoned this scenario as I did not want to use an inordinate amount of time on what started to look like a dead end.

  3. Brutal .Net: Start without window; run for 6 seconds; stop by killing the process.

    This scenario is is here for completeness. This is the most brutal method, which relies entirely on .Net functionality. When stopping the process, it simply kills it, without giving it any chance to clean up. Note that ping command does not display any summary when it is stopped in this fashion.

  4. Start without a window using .Net, run for 6 seconds, stop by attaching console and issuing ConsoleCtrlEvent.

    After finding a hint at the bottom of this thread, I finally bit the bullet and started the process without a window. When the time comes to stop a process, the parent attaches its console to the child, stops itself listening to Ctrl-C event and issues the event to the console, so that the attached child terminates. It then reinstates the Ctrl-C handler for itself and frees the console.

Resources

During the research phase, I got ideas and inspiration from the following posts and sites. Some them were dead ends, while others lead to something usable. The post in particular (the first link) was a real revelation, and is incorporated in the fourth scenario in the demo application.

Bitcoin Miners in Tray – Slim Universal Front-End

Posted on by
22

The Story

After using GuiMiner for a couple of months, I got tired of a few of its shortcomings, so I designed and wrote a program, which meets my needs exactly and can do the following:

  • Specify priority level for the individual miners
  • Specify processor affinity (even on Vista, where GuiMiner fails to do so)
  • Specify the backup pools
  • Can stop the miners nicely, by issuing Crl-C event, so that the miners have a chance to clean up. (GuiMiner simply terminated the miners, which resulted in NVidia display driver crash.)
  • Minimalistic GUI design
  • Be fully-configurable through a single XML file
  • Can run any miner, independent of its parameter signature, including rpcminer-4way.exe
  • Can run a set of programs prior to launching the miners – ideal for applying overclocking, using ClockTweak, or starting mining-proxy when when using getwork miner against Stratum pools.
  • Can keep the miners alive by checking their output and run state and re-starting them if they become unresponsive
  • Can auto-launch all the miners and hide itself in a tray with minimum fuss
  • Can be completely managed from the tray
  • Can watch the system for a set of conditions and modify miner state accordingly

Compatibility

I do not distribute any miners with this front-end. Indeed, it can support not only Bitcoin miners, but other long running command-line applications, such as Folding@Home client. This applicatin has been tested with Poclbm, RPC Miner, and Phoenix 2 Miner.

A note about Pooler’s cpuminer for Litecoins – Bitcoin Miners in Tray can run and control it, but due to two quirks, no output from cpuminer is displayed until it is stopped. Cpuminer directs all of its output to stderr, which BMT handles well. However, it does not finish output of a line when a line is written to stderr, thus all lines are dumped on the pipe at the same time when cpuminer stops and closes its streams. There is nothing I can do about it, as this needs to be fixed by Pooler.

CGMiner must be started with --text-only or -T parameter to disable ncurses and with --log 1 or -l 1 to log output every second. CGMiner suffers from the same problem as pooler’s cpuminer, which is not strange as it was forked from it. The problem is however only with the redirected stdout stream, while stderr works fine, outputting the lines as they are pushed into the pipe. Once CGMiner is stopped, all text on stdout will be dumped in one go. -l 1 seems to fix this.

Download

Version 6.00: – in progress
– Complete rewrite of the Process launching code, encapsulating the .Net Process class, so as to be able to peek on the stdout/stderr output pipe from the miners, solving the problem with some miners (pooler’s cpuminer) not showing any output until they exit. (Thanks to WebMaka for a lead.)
– All speeds in the balloon tooltip are now shown in the “dot” decimal notation, culture independent
– Added Speed column to the GUI list of miners.
– Added milliseconds to the log timestamp.
– Made config file path in the status bar into a clickable link, which would open Explorer at that path.
– Added restartEvery attribute to the miner node, allowing miners to be force restarted at certain hour intervals.
– Possibility to specify how the miner reports its speed in the list and the tooltip – K, M or G – through the new displaySpeedIn attribute. The total in the tooltip and GUI is always in M.
– Fixed a bug in hashrate-based keepAlive threshold calculation, which resulted in frequent false positives.
– Fixed a problem with disabled auto-scroll. Windows interop EM_GETSCROLLPOS returns scaled down 16-bit values even though it has a 32-bit Point structure at its disposal, resulting in erronous behaviour when the size of the RichTextBox content height exceeds 65535 pixels. A work-around has been applied, which alliviated the problem, though the text would not stay completely still.
– Fixed an off-by-one error in average speed calculation.
– The donation hint now points to Durrell Wildlife Conservation Trust fund-raising: 1DodoExzsNPvVRXFrgkKw6E259VjfUW8Kh

Version 5.10: [Binary] [Source]
– Added “Run with Windows” checkbox to the GUI and tray context menu.
– Improved positional handling of the output text box when autoScroll feature is disabled.
– KeepAlive Hits column now shows counters for each category.
– Reset KeepAlive Hits counters when a miner is manually restarted.
– Added threshold attribute to the checkHashRate keepAlive option. An average hashing speed during the keepAlive check interval, which is below this threshold will trigger miner restart.
– Added a small and unobtrusive donation hint to the main GUI screen. ;)

See the changelog for older versions and changes.

This program is represented at Bitcointalk software forum.

Donations

1DodoExzsNPvVRXFrgkKw6E259VjfUW8KhIf you find this program useful and use it on the daily basis, please support a good cause and donate to my Durrell Wildlife Conservation Trust bitcoin fund-raising drive

1DodoExzsNPvVRXFrgkKw6E259VjfUW8Kh.

Configuration

An example configuration file is included with the binary. Edit it to reflect your setup, prerequisites and miners and rename it to BtMinersInTray.xml

Screenshot

Bitcoin Miners In Tray

Bitcoin Miners In Tray - Balloon Status Tooltip Bitcoin Miners In Tray - Context Menu

The GUI is largely self-explanatory. Running miners are shown in green, stopped miners – in orange and mis-configured miners (wrong workingDir and/or application info) – in red. One or more miners can be started or stopped by marking the checkbox by their names. The program starts directly into system tray. Double-click the icon to show the GUI window. The probram can be minimised back to tray with (_) or closed with (X). Reloading config will stop all the running miners, read the new configuration and start the miners with any updates.

Almost all operations can be performed without opening the GUI window, through the use of the context menu and the status balloon tooltip.

Changelog and archived versions

Version 5.00: [Binary] [Source]
– Long-running prerequisites are now stopped when the program exits or the config file is reloaded.
– Prerequisites can now have keepAlive setting, watching for Process and Output conditions.
– Replaced custom-drawn checkbox-containing ListBox with ListView control. It is now multi-selectable, and without checkboxes.
– ListView box now contains detailed information about each miner’s configuration, including current priority and keepAlive hit count. Ideally it should stay on 0. If the value is growing, it means that you either have a misconfigured keepAlive condition, so that the function kicks in more than it should, or that you have overclocked your GPU too much and it keeps crashing the driver.
– KeepAlive now reports reasons for restarting the miner (in the log window).
– The application now targets .Net Framework ver. 4.0 and is built with explicit platform target set to x86.

Version 4.10: [Binary] [Source]
– Fixed a bug, where setting keepAlive to 0 would cause a crash
– keepAlive is back to being a child node of a miner, and can now be configured with interval and restart conditions: Process, Output, and HashRate.
– Fixed a rare occasion, when both the user and a watcher starting a miner at exactly the same time would result in two instances of the miner to be launched.
– Fixed handling of start/stop/restart watcher events to make it more resilient.
– Added killDelay miner attribute, so it is possible to tweak the time between Ctrl-C event is sent to the miner and the miner is deemed unresponsive to it and is forcefully killed.
– waitForExit attribute of a prerequisite execute statement is renamed to delay and can now specify a delay in seconds, in addition to special values of ‘no’ and ‘forever’.

Version 4.00: [Binary] [Source]
– Added locks around critical sections in miner start and stop routines. Addition of asynchronous watchers introduced a possibility of race conditions.
– Changed the colour of miner entries in the list and context menu: Green – running, Orange – suspended by watcher, Red – stopped. Misconfigured/invalid miners are no longer shown in the lists.
– Changed the way watchers are defined: condition and action parameters are now specified in the body of the watcher element. See example config file.
– Watcher check interval can now be configured on a per-miner level.
– Added two new watcher actions: RestartWithArguments and ChangeAffinity.
– Priority action is renamed into ChangePriority.
– Added three new watcher conditions: BatteryPower, TimeInterval and DaysOfWeek.
– Renamed Enabled miner setting to autoStart to better reflect what that setting actually does.
– Made averageWindow setting miner-specific.
– Application watcher can now specify multiple processes to watch.
– Tray icon is now visible while GUI is maximized (unless Stealth Mode TrayIcon is specified).
– Seconds are now displayed in the log timestamp.
– Breaking change to the config format: id, autoStart, keepAlive, averageWindow, and watcherCheckInterval are now attributes of the <watch> node, instead of being child nodes. This is done so as to make the config file less cluttered
– Breaking change to the config format: waitForExit is now an attributes of the <execute> node, instead of being a child node.

Version 3.00: [Binary] [Source]
– Introduced watchers that can observe such conditions as human activity or a running application and start/stop/change priority of the miners accordingly.
– Stealth config option which can either disable on-hover balloon tooltip or hide the tray icon altogether, in which case showing GUI is done by running the program again.
– Keep-alive setting is moved to individual miners (because of pooler’s cpumine’s faulty output mechanism). The default is now ‘off’.
– Total hashing speed info moved to status bar.
– Added possibility to specify any config file through the command line argument.
– Added possibility to open any config file through the GUI.
– Information about currently loaded config file is in the status bar.
– AutoScroll can be togged from GUI.
– Stealth mode and average window info is in the status bar.
– Fixed a bug, where user-stopped miner would sometimes be treated as dead and restarted.
– Fixed a bug, where the program would crash if no averageWindow was specified in the config (missing default).
– Improved stdout parsing heuristics to read correct speed info from CGMiner output and to avoid false positives.
– Brought most of the business logic under test, reordered some code, latest additions programmed using TDD.
– Various bug fixes as the result of unit testing.

Version 2.20: [Binary] [Source]
– Individual and total hashing speed is now shown in on-hover balloon tooltip in tray icon.
– Tray icon now has a context menu with the commands to open the GUI, stop all miners, reload config, start/stop individual miners, and exit.
– Average window for hashing speed calculation is now configurable.
– Checkboxes of the misconfigured miners are now disabled.

Version 2.10: [Binary] [Source]
– Fixed a rare bug, where an exception would be thrown if checkbox list redraw occurred at the same time as config file reload.
– Simplified routine that stops the miners, eliminating the need for briefly-flashing command line windows when starting miners.
– Added colour tagging of the output window: Green – program output; Blue – miner stdout, Red – miner stderr.
– Made output textbox read-only.
– Average hashing speed across all miners is now displayed in the GUI.

Version 2.01: [Binary] [Source]
– Fixed hiding of the icon in task switcher, when the application is minimised to tray.

Version 2.00: [Binary] [Source]
– Initial stable release

The first version was published at BitcoinTalk newbie forum, but was still very much a work in progress.

IXWebHosting cannot afford more storage space

Posted on by
1

All went well with IXWebHosting (where this site is hosted at the moment) until a couple of weeks back, when they started requiring some of the space-demanding content removed. They rationalised, that I am breaking their TOS by having some of the content, which is not accessible from the web-site. I conceded (there were a few files there that I kept as back-up) and removed that content. They then proceeded attacking the content, which constitutes a part of my web site here.

They demand that I remove the video files from my video projects, that I made available for download and still claim that I am breaking TOS by having large files on my site. This is ridiculous. Following this logic, they may as well start demanding that any site they host, which has anything more than pure HTML (or even that), remove any content.

They are boasting unlimited space, and that was one of the selling points, yet now they seem to be in real trouble and cannot afford buying a few more hard disks/servers!

I am now in the process of looking for another web hosting service, and I am unsure if IXWebHosting will refund a full year worth of prepaid services, that they are effectively terminating.

Steer clear of them!

EDIT: After some research, it seems that most web service providers have given a new meaning to the word “Unlimited” – it usually means: “About 10GB in total and individual files should not be larger than your average HTML. God forbid if they are over 1MB for then it starts smelling of file storage and you are violating our TOS!”.

Looks like I’ll be learning how to set up Apache securely on my local FreeBSD and redirecting the domain to my local IP. In the meantime this site will become crippled. Thank you, IXWebHosting :/

Another tree gets murdered

Posted on by
Reply

Eulogy

This tree ranged proudly over Lørenskog (which mean Laurent’s Forest) for hundreds of years. It saw the coming of men and building of small houses on the outskirts of Oslo. Later, when Lørenskog grew, it witnessed demise and slaughter of its sisters, until it was the only tree of old guard.

The residents in Georg Stangs vei 17B, the Tangens, have long gnawed their teeth at the tree, claiming that its top is slightly off-center and that it would topple, but that tree stood on, despite all the strong winds. Now the owners of the land at Georg Stangs vei 17C, where the tree actually stood forsook it and moved away. Quickly did the residents of Georg Stangs vei 17B move in and sent an army of hit-men to murder this proud forest giant.

Once they said to me, that the place for the trees is in the forest and not among people. But by this very same logic, it is them who should have been cut down by the teeth of a chainsaw, fore they are the ones, intruding into the realm of the forest, where there is not place for them.

As of now Lørenskog (Laurent’s Forest) commune should be renamed to Lørenvidde (Laurent’s planes). As Terry Prattchett wrote in the Science of Discworld, “Progress means smoke. Humans have surely made a lot of progress lately.”

File locator web front-end in PHP

Posted on by
1

When my file server resided in the Windows environment, I made use of the Everything search engine to index the files and to search for them both locally and through Everything’s built-in web server.

This latter functionality is what I wanted to replicate once I built the ZFS-based FreeBSD file server and moved to it. All UNIX flavours have the locate command, which will use a pre-built database to quickly find a string in file names and paths on your server. So, the obvious solution was to install Apache and PHP and write a web front-end for locate.

An alternative option is to use Solr, a Lucene search engine front-end. I, however, wished to have something simpler and custom-made. This was also a good opportunity to explore a new programming language.

I’ve never written PHP code before, as my main area is ASP.NET and C#, but learning the ropes of PHP was an enjoyable task and it is always good to learn another language. The result can be seen below.

The program will search for an arbitrary string in the locate database, optionally ignoring the case. Another option lets the user to restrict the search to the last segment of the path, thus avoiding flooding with nearly duplicate hits if the string is located only in the directory portion of the path. The program will also highlight the hits.

This is how the simple UI of the program looks like:

Update 1

I’ve added some more desired functionality:

  • the program can now update the underlying locate database through the web interface
  • it now accepts ‘*’ and ‘?’ wildcards in the search string and highlights the results appropriately
  • it can now give direct links to the located content
  • it can now search for string containing Unicode charachters
  • highlighting is made Lynx-friendly

Forcing database update involves running the update script as root, which will then su as user nobody. Apache (httpd) runs under a limited user www (or suchlike). To overcome this obstacle, I used a solution, suggested in this Stack Overflow thread:

  1. Modify update.launcher.c (code below) to point to the update script, which is typically located in /etc/periodic/weekly/310.locate
  2. #gcc update.launcher.c -o update.launcher
  3. #chown root update.launcher
  4. #chmod u=rwx,go=xr,+s update.launcher
  5. Place the program on your server and modify UPDATE_SCRIPT_LAUNCHER constant in the program
  6. Verify that LOCATE_DB_FILE constant points to the database file, so that the porgram is able to report the state of the database

Remember to change the value in SEARCH_ROOT constant, which limits the search location range.

If you want the program to display direct links to the located content, perform the following 2 steps:

  1. Create a symbolic link to the root of your searchable content, as defined in SEARCH_ROOT
  2. Update VIEW_SYMLINK_PREFIX constant to point to that symlink, relative to web root or relative to the locator.html placement. (If this constant is not defined, the program will not generate any links.)

There are a few caveats and assumptions:

  • There is no thorough error checking involved
  • Unicode search is always case sensitive

locator.html

Download 
<?php
//Configuration constants:
define(SEARCH_ROOT, "/zstore/");
define(UPDATE_SCRIPT_LAUNCHER, "/usr/local/www/apache22/data/update.launcher");
define(LOCATE_DB_FILE, "/var/db/locate.database");
define(VIEW_SYMLINK_PREFIX, "./zstore/");

$searchString = isset($_POST['searchString']) ? $_POST['searchString'] : '';
$ignoreCaseChecked = isset($_POST['caseIgnore']) ? 'checked' : '';
$lastSegmentChecked = isset($_POST['lastSegment']) ? 'checked' : '';

if(isset($_POST['clearButton']))
{
    $searchString = '';
    $ignoreCaseChecked = '';
    $lastSegmentChecked = '';
}
?>

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<style type="text/css">
span.marker
{
    background:yellow;
    font-weight:bold;
}
span.error
{
    color:red;
    font-weight:bold;
}
span.info
{
    color:blue;
}
</style>
<title>File Locator</title>
</head>

<body>
<table width="100%" border="0">
<tr><td width="80%">
<form action="<?php print $_SERVER['PHP_SELF']; ?>" method="post">
Search for: 
<input type="text" name="searchString" value="<?php echo $searchString; ?>" /> 
(wildcards * and ? are allowed)<br>
<input type="checkbox" name="caseIgnore" <?php echo $ignoreCaseChecked; ?> /> 
Ignore case<br>
<input type="checkbox" name="lastSegment" <?php echo $lastSegmentChecked; ?> /> 
Search in last segment only<br>
<input type="submit" name="submitButton" value="Go" /> 
<input type="submit" name="clearButton" value="Clear" />
</td><td valign="top" align="right">
<input type="submit" name="updateButton" value="Update file name database" /><br><br>
<input type="submit" name="stateButton" value="Show database state" />
</td></tr>
</table>
</form>
<hr>

<?php
if(isset($_POST['updateButton']))
{
    updateDatabase();
}
else if(isset($_POST['stateButton']))
{
    showDatabaseState();
}
else if(isset($_POST['submitButton']))
{
    $ignoreCase = isset($_POST['caseIgnore']);
    $lastSegmentSearch = isset($_POST['lastSegment']);

    if(updateLocatorIsRunning())
    {
	print '<span class="info">File name database is currently being updated.<br>Search results may be inaccurate.<br><br></span>';
    }

    $ret = array();
    $command = 'locate ' . ($ignoreCase ? '-i "' : '"') . 
		SEARCH_ROOT . '*' . $searchString . '*"';

    exec($command, $ret);

    $word = str_replace(array("?", "*"), array(".", ".+"), $searchString);

    foreach ($ret as $line)
    {
	if($lastSegmentSearch && !foundInLastSegment($line, $word, $ignoreCase))
	{
	    continue;
	}
	
	$find = highlight($line, $word, $ignoreCase);
	if(defined("VIEW_SYMLINK_PREFIX"))
	{
	    $link = str_replace(SEARCH_STRING, VIEW_SYMLINK_PREFIX, $line);
	    print '[<a href="' . $link . '">View</a>] ';
	}
	print "$find<br>\n";
    }
}

function showDatabaseState()
{
    if(updateLocatorIsRunning())
    {
	print '<span class="info">File name database is currently being updated.</span>';
	return;
    }
    else
    {
	clearstatcache();
	date_default_timezone_set('UTC');
	$dbtime = date("D, d.m.Y, H:i:s", filemtime(LOCATE_DB_FILE));
	print '<span class="info">File name database was last updated on ' .
		$dbtime . '</span>';
    }
}

function updateDatabase()
{
    if(updateLocatorIsRunning())
    {
	print '<span class="error">File name database is already being updated!</span>';
	return;
    }
    $command = UPDATE_SCRIPT_LAUNCHER . " > /dev/null 2>&1 &";
    exec($command);
    sleep(1);

    if(updateLocatorIsRunning())
    {
	print '<span class="info">Started updating file name database.</span>';
    }
    else
    {
	print '<span class="error">File name database updator failed to start.</span>';
    }
}

function updateLocatorIsRunning()
{
    $ret = array();
    $command = "ps -U nobody -o command";
    exec($command, $ret);
    foreach ($ret as $line)
    {
	if(strstr($line, "locate.updatedb"))
	{
	    return true;
	}
    }
    return false;
}

function foundInLastSegment($line, $searchString, $ignoreCase)
{
    $search = '/(?=[^\/]+$)' . $searchString . ($ignoreCase ? '/i' : '/');
    return preg_match($search, $line);
}

function highlight($text, $word, $ignoreCase)
{
    return preg_replace("/($word)/U" . ($ignoreCase ? "i" : ""),
                        "<span class=\"marker\"><b>$1</b></span>",
                         $text);
}
?>
</body>
</html>

update.launcher.c

Download 
#include <stdlib.h>
#include <sys/types.h>
#include <unistd.h>

int main (int argc, char *argv[])
{
    setuid (0);
    system ("/bin/sh /etc/periodic/daily/320.locate");
    return 0;
}

RegEx to match a substring after a delimiter

Posted on by
Reply

They say that if you have a problem and want to use RegEx to solve it, then you have two problems. So true! :)

My specific problem was that I wanted to search for a string within a substring after a delimiter sign, more precisely, in the last segment of a path. Here is an example:

/some/test_path/to/search/with_a_Test_file.txt

The RegEx, searching without case sensitivity for “test” should return a match only for the portion of the string after the last “/”.
All suggestions, which I could find on StackOverflow, concerned with matching the entire file name and not a portion of it, so I had to learn some advanced RegEx. Fast.

The answer was something, called “lookahead”, which is well explained at Regular-Expressions.info site.

The resulting RegExt string looks like some serious swearing in a cartoon bubble… :) Here is the code, which is accepted by PHP’s preg_match() function:

/(?=[^\/]+$)test/i

According to my (rather limited) understanding of RegEx, the first portion in the parenthesis, after the “?=”is the lookahead, which matched the entire file name after the last “/”, then comes the search substring, “test”, which operates on that result and, finally, “/i” is the switch, instructing a case-insensitive match.

Adding disks by label in ZFS and making those labels stick around

Posted on by
1

When I stared building my new file server, I decided to add the disks to ZFS vdevs by label and not by the device id, i.e:

#glabel label l1 /dev/ada0
#glabel label l2 /dev/ada1

After a reboot, those labelled disks suddenly started to show up as /dev/ada0 and /dev/ada1 again and the labels disappeared from /dev/label directory.

For the existing disks, I tried to offline each disk in turn and re-label it. A new problem turned up then: I could not replace the /dev/adaX offlined disks with the same labelled ones, as zpool gave an error of the device being already a part of the pool.

After some further searching, I found out that I had to zero out the first and the last megabyte of the disk before labelling it and replacing in zpool:

#dd if=/dev/zero of=/dev/ada0 bs=1m count=1
#dmesg | grep ada0
<read the block count value, subtract 2048 and provide the result to the seek switch below>
#dd if=/dev/zero of=/dev/ada0 seek=358746954
#glabel label l1 /dev/ada0
#zpool replace zstore /dev/ada0 label/l1

At this point zpool status was again showing labels. However, after the next reboot, the labels were gone again and I was pretty frustrated. Back to the search engine.

On page 3 of some discussion of this matter, I noticed two additional steps, which should fix the problem. After performing the steps above and re-labelling and re-placing the disks, I issued:

#zpool export zstore
#zpool import -d /dev/label zstore

The -d switch is what instructs zpool to read the disk references from a specific directory and it makes the labels stick around.

When I added subsequent new disks to the pool, I followed these steps to make the labels stick and to avoid re-labelling at a later point:

  1. Zero-out the first and the last part of each disk that will comprise the new vdev (especially important if the disk has been in use before and does not come staight from the factory)
  2. Label each disk with glabel
  3. #zpool add zstore raidz label/l5 label/l6 etc….
  4. #export zstore
  5. #import -d /dev/label zstore

And the labels never disappeared again.

This same procedure can be applied to labelling you ZIL and LARC devices.