Google AdSense

Sunday, July 31, 2011

Modified PHP bot for trading bitcoins on mtgox

General information about bitcoins (mtgox, mining, etc) here: http://bsdpunk.blogspot.com/2011/07/bitcoins-bitcoins-bitcoins.html

I've been messing around with bitcoin, and I found a bot at :
http://bitklein.com/bot.php

What I did was minimize the bot, and change it so it buys bit coins if they are less than 15 dollars, and I have over 60 bucks in my MtGox account.
If the bitcoins become worth over 24 dollars, sell all of them. I also have the mtgox bitcoin app on my phone in case I need to dump them.

function mtgox_query($path, array $req = array()) { 
   $key='yourkey';      //<----KEY! 
   $secret='yoursecret';   //<-SECRET! 
    
                   $mt = explode(' ', microtime()); 
                   $req['nonce'] = $mt[1].substr($mt[0], 2, 6); 
                   $post_data = http_build_query($req, '', '&'); 
                   $headers = array( 
                          'Rest-Key: '.$key, 
                          'Rest-Sign: '.base64_encode(hash_hmac('sha512', $post_data, base64_decode($secret), true)), 
                  ); 
                  static $ch = null; 
                  if (is_null($ch)) { 
                          $ch = curl_init(); 
                          curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); 
                         curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MtGox PHP client; '.php_uname('s').'; PHP/'.phpversion().')'); 
                  } 
                  curl_setopt($ch, CURLOPT_URL, 'https://mtgox.com/api/'.$path); 
                  curl_setopt($ch, CURLOPT_POSTFIELDS, $post_data); 
                  curl_setopt($ch, CURLOPT_HTTPHEADER, $headers); 
                  $res = curl_exec($ch); 
                  if ($res === false) throw new Exception('Could not get reply: '.curl_error($ch)); 
                  $dec = json_decode($res, true); 
                  if (!$dec) throw new Exception('Invalid data received, please ma 
  ke sure connection is working and requested API exists'); 
          return $dec; 
          } 
  $decoded=mtgox_query('0/getOrders.php?since=0'); 
  var_dump($decoded); 
  //print_r($decoded); 
 $seconds_wait = 40; 
  $truth =  TRUE; 
  while($truth){ 
          sleep($seconds_wait); 
          echo "----------------------------------------\n"; 
          $r=mtgox_query('0/getFunds.php'); 
          echo "USD TOTAL: ".$r['usds']."\n"; 
          echo "BTC TOTAL: ".$r['btcs']."\n\n"; 
          $decoded=mtgox_query('0/data/ticker.php'); 
  //      echo date('l jS \of F Y h:i:s A')."\n"; 
          echo "LAST:".$cur_last=$decoded['ticker']['last']."\n"; 
          echo "BUY:".$cur_buy=$decoded['ticker']['buy']."\n"; 
          echo "SELL:".$cur_sell=$decoded['ticker']['sell']."\n"; 
          echo "HIGH:".$cur_high=$decoded['ticker']['high']."\n"; 
          echo "AVERAGE:".$cur_avg=$decoded['ticker']['avg']."\n"; 
          echo "VWAP:".$cur_vwap=$decoded['ticker']['vwap']."\n\n"; 
   
  if(($r['usds'] > 60) && ($cur_buy < 15) ){ 
                  $amount= ($r['usds'] / $cur_buy); 
                  $amount = number_format($amount, 2, '.', ''); 
                  $price=$cur_buy; 
                  echo " PRICE: $price AMOUNT: $amount"; 
                  echo "Buying Bit coins to cancel transaction hit Ctrl C\n"; 
                  sleep(30); 
                  $req=array('amount'=>$amount,'price'=>$price); 
                  $decoded=mtgox_query('0/buyBTC.php',$req); 
                  echo "STATUS: ".$decoded['status']."\n"; 
  }                                
          if($cur_sell > 24 ){ 
                  echo "Selling Bitcoins\n"; 
                  $price = $cur_sell; 
                  $amount = $r['btcs']; 
                  $req=array('amount'=>$amount,'price'=>$price); 
                  $decoded=mtgox_query('0/sellBTC.php',$req); 
                  echo "STATUS: ".$decoded['status']."\n"; 
                  $truth = FALSE; 
          } 
  } 




You can get information about mtgox's api here, and about the bot: http://bsdpunk.blogspot.com/2011/07/bitcoin-software-for-mtgoxs-api.html


When I run this script I actually start a screen session, then type: php mtgox.php > mtgox.log
After that I open a terminal window and tail -f mtgox.log

Bitcoin software for mtgox's api

MtGox has this wonderful API: https://mtgox.com/support/tradeAPI

There's also a perl module for MtGox: http://search.cpan.org/~mndrix/Finance-MtGox-0.01/lib/Finance/MtGox.pm

And someone wrote this handy PHP bot, who I promise in the future I will donate some sort of bitcoinage too, because I ripped this thing apart and made my own, but he certainly made my life much easier:
Forum post:
http://forum.bitcoin.org/index.php?topic=31693.msg398512#msg398512
Actual Website to download File:
http://bitklein.com/bot.php

I'll post some of my modifications to the bot some time in the future.

Also if you're really interested in bitcoin mining, I invite you to take a look under the hood of the bitcoin miner poclbm
https://github.com/m0mchil/poclbm

Bitcoins bitcoins bitcoins

I'm going to defcon soon and it seem that there is only one lectrure on bitcoins. This saddens me. I use Mt.Gox to do all my bitcoin transactions. Which is to say convert my bitcoins into DOLLA DOLLA DOLLA bills. I am somewhat proud and somewhat ashamed to admit I am a bitcoin miner. As of right now I don't have any winning tips for you on how to be a miner. I will however introduce you to them.

Go here if you want to read the real version of what they are and how they work, instead of my dramaticized half fictional account: http://en.wikipedia.org/wiki/Bitcoin

Bitcoins are a currency based on cryptography, using various mathmatics to prevent shenanigans. Bitcoin miners, people who provide proof of work, for the bitcoins, can actually aquire bitcoins. And bitcoins can be turned into dolla dolla dolla bills. So to do this proof of work you rent out your proccessor, ok really you need to rent out your graphics card: ( https://en.bitcoin.it/wiki/Mining_hardware_comparison )

Woah Woah Woah, why are all the cards ATI..., well it turns out the function mostly used is sha256 and guess what, on ATI cards, there is a specific asm function for that....WAY TO GO Amd.


So get a client to cruch this cryptography and start cracking away. Use poclbm if you got graphics cards on the brain( https://github.com/m0mchil/poclbm ). Now if you want to use CPU's which is by all accounts is at least 50 some odd times slower you can still go for it, use RPCMiner for mac, and minerd for linux. Join a mining pool like deepbit and start crackalackin a way.

If you join a mining pool the rewards can come very fast.......but with very fast, come very small rewards, lol.

CAVEATS:
OH AND ONE HUGE CAVEAT. This will eat electricity like a mother fucker. No joke, if you live in (Worse case US scenario ) Hawaii, have two rigs, with 2 graphics cards a piece your electric bill can go up over 200 dollars what it normally is. LOl that math is bad and terrible. For real math ( http://www.pcper.com/reviews/Graphics-Cards/Bitcoin-Mining-Update-Power-Usage-Costs-Across-United-States ). This also makes crunching solely on cpus, pretty much a no gain death trap, by my understanding.

Updates to Contest and Goings on (Defcon)

If you have a submission for the contest find someway of getting it to me, whether that be bsdpunk at the gmail, or a comment to your github, or whatever your preferred method of delivery is. I'll announce a winner soon.

In other news, I will be at defcon, if you would like to befriend/harass me. So give me a tweetage @bsdpunk if you'd like to meet up.

Sunday, June 12, 2011

#devel contest draft rules

It's contest time again, and more particularly, it's contest time for the #devel channel on irc.malvager.net

Draft Rules for the #devel contest of irc.malvager.net:


Create a tool that will either allow or otherwise encourage people in #devel to collaborate in code writing and or sharing. The platform/language can be any (within reason; if you’re not sure, ask). It will be judged subjectively by both bsdpunk and psycho on both functionality and creativity. The first place prize will be a bsdpunk t-shirt and an Arduino (or Arduino clone). The second place prize is an Arduino (or clone). If god willing a third place prize, tbd (this one is not written in stone).

Contest will be over on the 20th of July.


Draft rules are subject to change for the next week, keep updated by looking at this blog(rules will only change by minor stipulations, the spirit of the contest will remain the same).

Sunday, May 29, 2011

The blog has been around for a wee bit. And will be around for a wee bit more.



I got my first check from google, for running ads on my site. My first blog post, for this particular blog, was on Tuesday, January 22, 2008. That check, is a reflection of from then, until now. I told my girlfriend that I had gotten the check and she said that was cool, and I was like not really, 3 years of work, translated into 73 dollars didn't seem good to me. And asked, well isn't it "free money", from a labor of love?

Hmm… Had to think on this one for a moment. It certainly didn't start out that way. I started writing the blog, thinking I could sell T-shirts and get advertisement revenue, by attracting teenagers with a disposable income. This certainly isn't what happened. If you will waltz down memory lane with me for a paragraph I'd appreciate it.

In the first year, I started it to make money. Which in retrospect was dumb. I got an account on a forum site, hackforums.net. Which turned out to be the right demographic, but people searching for the wrong knowledge base. For the first year that site was 70% of my traffic with the other 20 - 30% coming from google search results. I worked hard to be someone on that site and eventually I was even a moderator for it. I busted my ass that first year, writing the majority of the now 433 posts on this blog, being written in that time. And I tried to sell t-shirts, even had a photo shoot, lol.

I really began exploring other creative mediums as time progressed, the blog gave me a vehicle and confidence to do so. I wrote some, hmmm, would you call it music? Eventually finishing an "EP" which you can still download from last.fm.

I'm doing lot's of other little things. That are various and sundry labors of lover perhaps. In fact I'm microblogging right now at http://ramblings.malvager.net. I do a fiction blog that most certainly is a labor of love. And other side projects that may fall into that category. And another goblin majick album coming up too.

So what is it now, obviously I'm still blogging, and I'm not making money off of it. I've got about 5 posts planned out as of right now, don't know whether they all get released next week or next month. Is it a labor of love? It must certainly isn't. Is it to make money? Of course not. All I can say for sure, is it's a force of habbit.

irc.malvager.net, is my only continuously kept updated labor of love, and for me it's more of a money sync than a labor of love, because two(kye, psycho) of the GREATEST people, and damn good hackers too, help me maintain it, while I just pay a chunk of the server bill. Seriously, love you guys; in the words of Jesse Dangerously, I couldn't make a move without you guys, y'all my legs.

Thursday, April 7, 2011

Last.fm I must leave thee.

I really like last.fm, I like what it keeps track of. That's what I like about it. I don't like it's algo, the math they use to determine what songs you like. Yeah I listened to ICP in the 5th grade...no I don't need every single song I listen to today to be from Psychopathic Records. At one point on my "Reccomended" I could only get comedy because I liked a few comedians tracks, and a lot of nerdcore. ::facepalm::

Do not need.

So I don't care, they keep track of what I'm listening to, that's what I use the service for, and for the create your own station by tag or whatever the option is called. This station only mildly has to do with that tag, which for better or worse is usually better than the radio....wait, not if you live near/in Nashville. We have 91.1...at least for a little while. Losing that station is going to suck.

I digress. So I went to listen on my phone today, to Last.fm and sure enough there's a pay wall for mobile devices now. I did a double take...seriously. I started poking around there website to see what the deal was, and wonderfully enough there is an export in XML format, for your loved tracks.

I took the chance, and indeed Paul Simon was right. I hopped on the Bus Gus. So I grabbed that file. I decided to make it more readable for non geeks, so I did this to put a more readable version in a file called titles:

cat bsdpunk_lovedtracks.xspf |perl -pe "s/.*<title>(.*)<\/title><creator>(.*)<\/creator>.*/Song: \1 Artist: \2/gi"> titles


Yeah yeah I know, don't parse XML with regex, it will anger Chthulu. I still don't know what you are supposed to use to parse it without regex, and even if I did I probably wouldn't use it for this, it took me less than a few minutes to write.

Also does that snippet of code help...no not really. I mean that's to give you something that is human readable to other people, you know...normies, so you have a:

Song: Twenty-Three Artist: MC Lars
Song: Listen Close Artist: MC Frontalot
Song: In Arrears Artist: MC Frontalot
Song: Why So Nerdcore Artist: MC GoblinMajick


What will I ultimately do with this xml file...obviously not this, unless I just wanted to paste it up somewhere like a web page.

I think I actually want to build my own algo. In some way it can be run independently, of pay services that actually provide the music, like last.fm.

How will I do this...I have no idea. It's 4 in the morning, and I have to be at work in a few hours, I just couldn't sleep.

Until then I will use Pandora on my phone( evo shift ). My very awesome phone I am very much in love with(not sarcasm).

Monday, April 4, 2011

Read this, for his name is fishmeal

Success and Farming vs Mining
Read this, and if you don't read it, then the tl;dr I got out of it, is quoted below.


This doesn’t work. Your idea sucks. No, I’m not calling you stupid — my idea sucks, too. All ideas suck, because they are just ideas. They’re worth nothing.

My success is because I worked to make the idea real. A lot. All my life. Starting when I was 12, I learned to program, and I’ve programmed every spare moment since. I didn’t become a millionaire until I’d worked at it for eighteen years. There was no genius idea I had. I just kept working, hating what I did before, and working some more to make it better.


I've been thinking about how I should focus, on what I should be doing, who I should be working for, and what I should be learning. It was nice to have a lighthouse in the fog. I know where I'm headed, but the occasional reminder never hurts.

Thursday, March 31, 2011

Donate to Japan, Keep Checking on GeoHot and Keep it Real programming

I really appreciate this site because the only image is the center one(well the center image is a bunch of images put together), but the rest of the "logos" are randomly created through javascript. http://next.do/, I used the idea, and quite honestly borrowed a lot of the javascript to create something for my girlfriend.

This week though I couldn't afford to give much I did give to one charitable japan cause, one because I like 8 bit music and two because....well there was a god damned earthquake:

http://truechiptilldeath.com/japan/

You get a SHITLOAD of tracks for whatever you donate....if you do donate small like I do, I suggest you also donate the cost of the credit card transaction, the extra 83 cents or whatever isn't going to kill you.

And it looks like the first round of donations for geohot(PS3 hacker) are closed, but keep abreast of the situation Geohot's blog.


And if you need your daily dose of reality, in the programming realm, take a look at this Use your brain.


Ok no more shitty link posts....ever again...I promise...until next month anyway.

Using Javascript in the address bar

One easy way to run javascript in your browser is to, simply type:

javascript: and then a javascript function

So for example, if you want to change the term Christian Business into scams, just paste this into your address bar.

javascript: document.write(document.childNodes[1].innerHTML.replace(new RegExp(/Christian Business/gi), "scams").replace(new RegExp(/Christian Business/gi), "scams"))

To test this you could go here: http://www.google.com/search?hl=en&q=%22Christian+Business%22&aq=f&aqi=g10&aql=&oq=

And then just paste said javascript into the browser bar.

Wednesday, March 30, 2011

The questions and the aftermath

After the interview that changed me from believing I was masterful wizard of the cli, into realizing I was more like Schmendrick before he met the Unicorn. I thought I would share some of the questions, and what I've been doing to improve my knowledge. This isn't a comprehensive list of all the questions as the interview is mostly a blur, as mentioned before I usually knock technical interviews down like dominos. This one I didn't and I can only blame my soft knowledge of linux, which I greatly exaggerated in my own mind. For the sake of brevity, and sort of for accuracy as well, when I got one right he seemed to nod, and write the extent to which I felt the mastered the question down, so I'll say nod for those, and for the ones I got wrong, he just put a tick mark next to that.



Preliminary non Technical Questions


Can you pass a drug test?

Yes.

Are you sure?

Yes.

It's a hair test.

I nodded.

THEY WILL FIND A HAIR.(I'd like to note he did in fact raise his voice)


I'd like to note that during the interview I was pretty "clean" looking. I had shaved, cut my hair, was wearing a shirt and tie, however any other given day perhaps this question would require more than one pass, as I do usually sport long hair and neck beard, wear t-shirts, and generally look of malaise. He could have been reading my body language or he could have just not wanted to deal with a 2 hour technical interview that day, or with someone who wasn't going to pass anyway. I don't know.



He asked if I could pass a credit check.

Yes.
He asked if I could pass a background check.

Yes

The Technical Terror Interview


Like I said these are the questions I remember, as close to the order I remember. The interview was two hours, so obviously quite a few are missing. Also I started by adding that some of the facts on my interview were padded or inaccurate, because my recruiter had rewritten it. I went over these with him so that if necessary we could cancel the interview. However, he didn't seem to mind the areas I was weak in.






Have you ever compiled the linux kernel?


I have it's been longer than 5 years ago, it was for a driver issue or something like that, if I remember correctly. It's been a long time.

nod






What are the tools required to compile the linux kernel?


Like I said it's been a long time, I don't really remember, I'm sure gcc is involved at some point.


tick






Describe the linux boot process after it is handed over from the bios


Note: Panic set in, I lost my shit, why didn't I know this, I know I should know this.

So I hate to admit this but I don't know,if I had to guess it would go to the boot manager, then probably the kernel, then the kernel modules then kudzu at some point maybe.*sigh*


tick




This is probably where the gaps start but I regain conciseness somewhere about 30 minutes-ish later, I know I said something about apache during this time period that seemed to tickle his fancy, but that's about all the good that thirty minutes had



What editor do you use?....Oh well of course I know what you use.

Vim?

I'm guessing it's the only real editor to you?

Well it's my favorite, but I have respect for my emacs brethren

At this point, a calm in the storm, we had an editor conversation where we both seemed to agree on a lot of it. Mostly it was about how to teach those new to vim how to use it. We also did a little nano slamming.




What is a zombie process?

A process that has lost communication with it's parent and probably needs to be killed.

Half Tick, Half Nod, slight writing.


He explained that the parent process to had died and he asked me how to kill it.

Probably a kill -9.

Hmmm...

tick

Just a kill or kill -4






Do you know the difference between kill -9 and kill

I believe a kill -9 is forcing a program to close and a kill is more like asking the program to close

Not sure if I got a tick or a nine





I feel quite defeated at this point.






Can you name any other signals?


no







Can you tell me about the runlevels


Not a whole lot I know there was for halt.(I do know more about the runlevels but at this point my brain was on a tactical retreat, and if the doors didn't have retinal scanners I would have run from the building like a little girl....ok ok the retinal scanners were only to get into the room, it was the social contract that kept me there.)

tick





Another black out.






What is cross site scripting?


I was defeated, I was non caring. I answered with the enthusiasm of a mental patient on so much thorazine, his drool would drool.


I don't know, javascript, steal cookies something.




It seemed that seeing my interested fade, had somewhat diminished the perverse pleasure he may have been taking in the interview. The questions got a little stranger.

What's the farthest you could get in a MySQL injection attack?


I start pontificating on UNIONS and LIKE's and REGEXP's and he knew he had me back in the game.


No tick or nod, I don't think this was on his questions sheet





Do you know what ARP poisoning is?

Boy Howdy, I answered this with aces, explaining the arp protocol, and what programs to use to do this. How to do this on linux, windows, or how to write something in perl that would accomplish the task. Then I told him how you could even fake an ssl cert.


This peaked his interest. He said "But wouldn't there be a nag screen saying the cert wasn't valid. Certainly I wouldn't log into my bank if such a screen popped up"

"Neither would I, but would she" I pointed to the secretary that was several glass walls between us.


NOD.




He started a network question: If I had a subnet mask of 240...

Before he finished I said "16 hosts, 16 networks"


Nod


We talked about VLSM for several minutes


We got into an argument about UDP, I had this half thought out theory about offloading the handshake to the processor to conserve bandwidth(yes I realize I was wrong now, and did admit it in the interview). It was terrible, I argued with an interviewer with a half baked theory.


He congratulated me on knowing a lot about networking anyway, and said a lot of linux admins were weak in that area.






Name three ways to restart a linux box?

Again I had lost interest. "Umm, shutdown -r now, and probably one of the runlevels"

I can tell you started using linux a long time ago, most newer guys don't know shutdown -r.

This was his final and what I considered second compliment.





He asked me if I had any questions. I said I felt dumb. He said "Sometimes an interview is just to let you know, you aren't qualified for the job."




What am I doing about?



Well I purchased Linux Kernal in A Nutshell for the kindle app on my android and I've been reading it. I'm going to try and complete a http://www.linuxfromscratch.org/ tutorial this weekend. I have bookmarked a webpage that is the equivelent of man page for linux signals, and the wiki page for run levels of linux. I took this interview because my contract was running out on a web programming job(perl, but in a windows enviroment). I'm also going to get more organized and focused on what I need to be learning.



I'm also not going through recruiters unless I must. Unless it's this guy, The Antipimp , he's a friend and trustworthy. I've since learned my current recruiter(not the one that set me up on my interview/lied on my resume), is making around .7 what I make on every hour. So for the sake of arguement and easy math(ie these numbers are false), if I'm making 30 dollars every hour, my recruiter is making 18 every hour.*




So that's where I'm at. If you have suggestions on what to learn/read/do, please let me know.



* I just redid the math and it's actually closer to .8.

Tuesday, March 29, 2011

The Linux Job Interview from hell.

*I've been an (very) amateur linux enthusiast since 1999 and a profesional linux admin for 5+ years*


**Hi Redditors and Hacker News people, I got a huge amount of comments on specifics about the interview, like what the questions were, etc, I have since written that information here http://bsdpunk.blogspot.com/2011/03/questions-and-aftermath.html . **

So what do you do, when you've been bested? When you have been so wholly destroyed in an interview, it shakes you to your core. When you get about thirty percent of the questions right, and you can't blame anyone but yourself. It was a face to face interview, and when the interviewer saw you struggle, and you were on the right track he helped, in the capacity he could without giving you an answer.


A two hour face to face interview with only technical questions. No way to ease out of this one, no excuse. "The multiple choices was obscure", "The wording was wrong", "When would I ever need to know that." All those justifications were thrown out the window. It was no use to hold on my pride.
I had my skin ripped off me. I thought I was somebody, I thought I new Linux like the back of my hand. But more importantly I recognized, for the first time in the long time, that I knew very little. I'd like to blame the product of my environment. That for the past 6 or so months, I've had no coworkers, or friends in the field, to truly talk to, or compare notes. Only the occasionally tweet exchange with strangers. Or that I've been working in a windows environment for about the same amount of time. But this isn't the fault, this isn't why, it was my arrogance.

I've always done well in interviews, all types of interviews, but when it comes to tech interviews, they've always been like candy from a baby. Not this one. So I am going to lay some things down for you.


What did I do wrong? To start with, which is only the beginning. I didn't review the basics, you know stuff you learned in school, or just dicking around with Linux. So simple questions about the kernel boot process, stuff I hadn't though about in a while, because I was used to just booting machines in the cloud and setting them up with automated scripts I wrote, hadn't thought about the boot process in a while. Stuff like runlevels and signals, I know what they are, I know how to Google them, but I don't have exact memories, I know kill -9 is force and normal kill and ctrl-d are ask the program to quit....can I explain it more technical than that, apparently not.


I missed questions I knew but forgot the answers to, like the difference between asymmetric and symmetric encryption. I just forgot the terminology, cause I didn't even glance at a book; I had this thing in the bag I thought. If he would have asked me to describe PKI, I would have pwned that question like Charlie Miller, chowns a mac. But he didn't, I was lost.


I missed questions I didn't know the answers to, and had thought were irrelevant....until he explained the answers to me.
I even asked him to ask me questions on things I thought I knew, like rsync, and was humiliated worse than Ben Stiller in any of his movies. After being grilled on why I would use the v, I knew it was over.


After all this frustration, I had the audacity to get in an argument about UDP which I hadn't fully thought out, and of course my argument was crushed, and ultimately wrong to begin with. (I assure you at this point I had already crashed and burned so hard this was not a deciding factor in me being hired).


Did I go in with the wrong attitude? No, I don't think so. I just wasn't technically skilled enough for the position, I had a belief that, I can survive in any position as long as I'm given enough lee way for a couple of weeks so that didn't scare me off.


I'm going to say 99% of the failure of this interview was my fault, I wasn't skilled enough, and I didn't prepare, as I should. I would like to fault someone else for the other 1%, a recruiter who buffed my resume without my knowledge, sending me in the dragons den with embellished, if not downright false information, with my resume.... rewritten to serve his needs.






I'll follow this up with how I am going to remedy my skillset/interview/recruiter policy in the next few days.


EDIT: Ctrl-D is actually end of file, thanks to the peeps at hacker news. Please keep reporting inaccuracies and I will fix them as they are found.

Wednesday, March 9, 2011

Favorite way to regex a file using perl

Sometimes you want to do some magic fu on your file, while it's not loaded in a vim buffer. This is what I use with perl. I know there is probably a way to do this with sed, but I couldn't find it quickly enough. Most of the time when I do a search and replace out side of vim I want to replace something that's already in the file, with itself in a different position. For example:

I want to replace variablewordspace with variablewordtime, so in the file var.txt I have:

variablewordspace
hammerspace

So what I would do is:


cat var.txt | perl -pe 's/(\w+)space/\1time/'

which would give me the output:


variablewordtime
hammertime



For a brief description:
You are display the file var.txt, through the cat command. Then using the pipe (|) you are passing that to perl, with the switches p and e. Switch e allows you to run from the command line without explicitly writing a perl program, while switch p allows you to run a program against every line on standard input, and prints whatever is in $_ after each line. A search and replace using s/// with the search term in the first set of slashes

/(\w+)space/

The parentheses mark that as a group.

The \w+ means any number of word characters, as where the 'space' just represents the word space.

In the last set of slashes is what you want to replace it with.

/\1time/

\1 represents what is in that first set of parenthesis.

And time of course, represents the letters 'time'

Thursday, January 13, 2011

Simple Stupid Linux Tricks ssh port forwarding

At work I'm firewalled in, and I'm on a linux server...however they allow traffic over port 22. So it's really easy to get to meebo. You can do this with most sites, however if they require more than one port you'll need a more complex solution.

I fire up cygwin and type this in.(You can use any ssh client though)


ssh -L 8888:meebo.com:80 root@wereboobs.com -N

ssh creates a tunnel from me to wereboobs, which the proxy doesn't care about to meebo which the proxy does(ironic, no?). So now if I type localhost:8888 into my browser I can get to meebo without going through the web proxy. Bonus...the web traffic from meebo is encrypted until it reaches wereboobs.

Thursday, December 16, 2010

Grab your delicious bookmarks before they're gone

Rumors that delicious is being closed by yahoo, made me want to grab my bookmarks, Someone posted this on twitter

curl https://user:pass@api.del.icio.us/v1/posts/all


via :adactio

Which is nice but gives you all that xml gunk.

This gives you a nice list in a file bookmarks.txt

curl https://user:pass@api.del.icio.us/v1/posts/all | grep -E -oh 'http:\/\/.*?/"' | grep -E -oh 'http.*?/' > bookmarks.txt

Tuesday, December 14, 2010

Pull the alt / title tag from xkcd picture from command line

How to pull the alt (actually the title tag), from xkcd with the command line

[root@queerlikerice ~]# curl xkcd.com | grep -E -oh 'title=".* alt=' | grep 'title' | grep -E -oh '".*?"'
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 8375 100 8375 0 0 137k 0 --:--:-- --:--:-- --:--:-- 538k
"And if you labeled your axes, I could tell you exactly how MUCH better."



Bash cli:
curl xkcd.com | grep -E -oh 'title=".* alt=' | grep 'title' | grep -E -oh '".*?"'

Monday, November 8, 2010

Mounting a blue ray iso on linux, to play in vlc

Recently downloaded a blu ray movie on my seed box, went to play it and I could mount it in ubuntu but the default setting still would not let me watch the movie in vlc. However after a quick google search I went here:
http://ubuntuforums.org/showthread.php?t=1209981

And sure enough the guys advice which was to mount with:
sudo mount -o loop -t udf pathtobluray.iso /media/cdrom

Worked perfectly. Once mounted navigate to the directory with vlc /dev/cdrom/BDVM/STREAM/*.m2ts

I put an asterisk because I am not certain the m2ts is the same name on every blu ray.

Using perl to create hash tables like your mysql tables

The subroutine Below will take an array of table names and put the columns in a hash, then grab rows and match said rows with said field/column. So that $tableData('salesdatafirst_name') would be the value for the first name column in the tables sales data, in the row based on your condition in the second query of the subroutine.


sub get_table_data {
foreach(@_){
my @ColumnNames;
$data_name = $_;
$dbh = DBI->connect($connectionInfo,$userid,$passwd) or print "Couldn't connect to database: ". DBI->errstr;
$sql = "SHOW COLUMNS FROM $data_name";
$str = $dbh->prepare($sql) or print "Couldn't prepare query: ". DBI->errstr;
$str->execute() or print "Couldn't execute query: ". DBI->errstr;
while (@querytwo = $str->fetchrow_array()) { #loop thru results
$tableData{$data_name.$querytwo[0]}='';
$tableHashHelper = $data_name.$querytwo[0];
push(@ColumnNames, $tableHashHelper);
}

$sql = "SELECT * FROM $data_name WHERE condition = '$yourcondition'";
$str = $dbh->prepare($sql) or print "Couldn't prepare query: ". DBI->errstr;
$str->execute() or print "Couldn't execute query: ". DBI->errstr;
while (@Q = $str->fetchrow_array()) { #loop thru results
for ($i=0; $i<=$#Q; $i++) {
$tableData{$ColumnNames[$i]} = $Q[$i];
}
}
undef(@tableColumnNames);
$dbh->disconnect();
}

Saturday, November 6, 2010

Retrieving Exif data, or Where she at bro?




Geolocation data in some cameras and many smart phones is embedded in the photos that these devices take.


View Larger Map


**Technical information**

The incredibly simple script **must install Image::ExifTool**
Image::ExifTool can be installed rather easily, through cpan.

use Image::ExifTool qw(:Public);
my $info = ImageInfo("$ARGV[0]");

foreach (keys %$info) {
print "$_ => $$info{$_}\n";
}

run it like this:
perl meta.pl test3.jpg

Output for this looks like:

GPSLatitude (1) => 43 deg 49' 42.60"
GPSLatitude => 43 deg 49' 42.60" N
GPSLongitudeRef => West
GPSTimeStamp => 01:00:12.48
GPSLatitudeRef => North
GPSLongitude => 79 deg 7' 24.00" W
GPSPosition => 43 deg 49' 42.60" N, 79 deg 7' 24.00" W
GPSLongitude (1) => 79 deg 7' 24.00"
lane@skullfuckerthemagnificent:~/Pictures$ perl meta.pl test27.jpg
XResolution (1) => 1
ImageWidth => 600
Model => iPhone
ExifImageHeight => 1600
GPSLatitude (1) => 43 deg 49' 42.60"
ResolutionUnit => inches
ColorComponents => 3
BitsPerSample => 8
GPSLatitude => 43 deg 49' 42.60" N
MIMEType => image/jpeg
FileType => JPEG
GPSLongitudeRef => West
ResolutionUnit (1) => None
ExifToolVersion => 8.25
FilePermissions => rw-r--r--
JFIFVersion => 1.02
Directory => .
FileName => test27.jpg
ImageHeight => 800
XResolution => 72
Make => Apple
DateTimeOriginal => 2009:04:08 01:00:13
GPSTimeStamp => 01:00:12.48
CreateDate => 2009:04:08 01:00:13
YCbCrSubSampling => YCbCr4:4:0 (1 2)
ExifByteOrder => Little-endian (Intel, II)
FileModifyDate => 2010:10:22 07:43:26-05:00
ExifImageWidth => 1200
ColorSpace => sRGB
EncodingProcess => Baseline DCT, Huffman coding
FileSize => 58 kB
YResolution => 72
Aperture => 2.8
GPSLatitudeRef => North
GPSLongitude => 79 deg 7' 24.00" W
GPSPosition => 43 deg 49' 42.60" N, 79 deg 7' 24.00" W
YResolution (1) => 1
FNumber => 2.8
ModifyDate => 2009:04:08 01:00:13
GPSLongitude (1) => 79 deg 7' 24.00"
ImageSize => 600x80

and if you have linux, mac, or your operating from within cygwin on windows this helps:

perl meta.pl test3.jpg | grep 'GPS'

Output for this will look like:

GPSLatitude (1) => 43 deg 49' 42.60"
GPSLatitude => 43 deg 49' 42.60" N
GPSLongitudeRef => West
GPSTimeStamp => 01:00:12.48
GPSLatitudeRef => North
GPSLongitude => 79 deg 7' 24.00" W
GPSPosition => 43 deg 49' 42.60" N, 79 deg 7' 24.00" W
GPSLongitude (1) => 79 deg 7' 24.00"


GPSPosition is the one your looking for you will need to trim it for google maps(IE 43 49' 42.60" N, 79 7' 24.00" W).


Most photos that get uploaded to the intarwebs get altered by resizing software or watermarking software or what not. When these photos are altered, many times thier EXIF information is cleared and replaced with information about the software that has done this(ie imagemagic GD). So most photos are not very useful. Sites that might be useful are probably twitter related image storage sites, simply because they consider stuff like geotags valuable.

Using DBI putting field / column names in a hash table

use DBI;
use DBD::mysql;
use strict;
use warnings;

my $database = "ci";
my $host = "localhost";
my $port = "3306";
my $tablename = "happy";
my $user = "root";
my $pw = "wouldntyouliketoknow";

my $dsn = "dbi:mysql:$database:localhost:3306";
my $connect = DBI->connect($dsn, $user, $pw);

my $query = "SHOW COLUMNS FROM $tablename";
my $sth = $connect->prepare($query);
$sth->execute();
my %hash;
my @ary;
while(@ary = $sth->fetchrow_array){
$hash{$ary[0]}= '';
}
my $key;
my $value;

print "\n";
while (($key, $value) = each(%hash)){
print $key.", ".$value."\n";
}

Same as the perl script before, but using DBI module instead of Mysql Module.

Loading a hash with the field names from a mysql table using cpan's Mysql;

#!/usr/bin/perl

use Mysql;
use Data::Dumper;

$host = "localhost";
$database = "ci";
$user = "root";
$pw = "wouldntyouliketoknow";
$table = "happy";

$db = Mysql->connect($host, $database, $user, $pw);
$fields = $db->listfields($table);
@hammer = $fields->name;
%hash = map { $_ => 1 } @hammer;





So you can load up a has with the field names on one side, so you can have variables like $hammer{primary_id}.

Cygwin, Where's My hardrive?!? interrobang

/cygdrive/~driveletter~

This is more a note for me, lol. New Job has windows computers, yuck.

Wednesday, May 26, 2010

Getting Back



I started looking at the C Sound book yesterday. I knew instantly I needed to learn something new, something both technical and artistic, I haven't quite nailed down what I'm going to do, I do know the general direction I am going. I'm rebooting and reorganizing my experience idea. I was working with arduino and next to have sensors that tell what your mood is and try to change it(maybe for the better). I still love the idea, but I am going to add a human component. I have a lot of hopes of what it will be, and would love to share them with you but I feel if I do so and my plan never comes to fruition, then I will feel even more the fool. So to get in the right mindset, I am getting back in touch with my skiddie roots. I rode an electric skateboard to work today, I safety pinned anti Microsoft logos on my bag, and a cDc logo as well, and other such non sense. Even though physically and mentally I really don't feel that well, knowing I am starting a new project has rays of optimism and excitement. I will hopefully be posting here more, I will be posting on fucking fiction, most assuredly.

Thursday, May 20, 2010

Wednesday, May 5, 2010

Nashville Flooded

My hometown is under water...zomg. Pictures

How to help.

Tuesday, April 6, 2010

New posts and my feelings of purileness

I feel that the new posts, since my absence have at best been juvenile. I apologise for the poor content. I planned to have a big to do when the blog turned two, however that didn't happen, mostly because of a turn in my health and stress at my job/life as well as other obligations. I apologise for this. I will at some point in the future have a HUGE DEAL, to make up for it. However, as I am still in poor health, and my work and finances demand my attention elsewhere I will try to appease with what I can. Expect new content on http:\\fuckinfiction.wereboobs.com soon. And expect better posts on here.

One of the reasons for the ubuntu posts is because I was a hardcore fedora fan, however after using there test repositories and 13 alpha, I had so many crashes and data loss....I finally went to the darkside...yes, oh yes, I tried arch. I know it's all the rage. And all the cool kids are using it. But my problem with arch was many. So I went with Ubuntu, please forgive me.


Also, here's me straight razoring my head.....yes, I am an idiot.

Real Men Straight Razor their hair






Thanks,

Dusty C.

Ubuntu / Gnome change your wallpaper every 10 seconds

Ok so assuming you have a folder named Wallpaper in your home directory which contains numbered .jpg's(in my case 385) you can use this to change your wallpaper every 10 seconds. You can adjust how often it changes by changing the sleep number on the 5th line of the script and you can adjust the number of wallpapers you have on the sixth line of the script by changing the number 385. I originally had made this as a cron job rather than a constantly running perl job which I run in screen (screen perl wallpaper.pl...then ctrl + a + d). However I was having some trouble getting it to work with cron, and the outside sources I used to help figure this out where at a loss so here it is.


#/usr/bin/perl
use strict;
use warnings;
while(1){
sleep(10);
my $numberW = int(rand(385));
print $numberW;
my $wallpaper = "gconftool-2 -t string -s /desktop/gnome/background/picture_filename /home/lane/Wallpaper/".$numberW.".jpg";
system($wallpaper);
}


Now you may wonder how you get neatly numbered wallpapers. Well I wrote a script for that as well and it works as long as none of the wallpapers are already named as numbers already. Remember to change the @files dir to the directory where the pictures already exist non-numbered, and to put the $newfile directory where you want the new files to go. It is not advisable that these be the same place.


#!/usr/bin/perl -w
use strict;
use warnings;
my @files;
my $it = "207";
print $it;
@files = ;
my $counts;
my $newfile;
my $file_number = "0";

while(1){
foreach (@files) {
my $file_regex = qr!(\d+\.jpg)!s;
if ($file_regex =~ @files) {
$counts = $counts + 1;
print $counts;
if ($counts == scalar @files) {
print "fuck you";
}
if ( $counts == scalar @files ) {
exit();
}
}
my $loopy = "1";
while($loopy){
my $file_regex = qr!(\d+\.jpg)!s;
if ($file_regex !~ @files) {
$newfile = '/Users/lane/Downloads/tat/ta2/'.$it.".jpg";
rename($files[$file_number], $newfile );
++$it;
++$file_number;
if( ! -e $newfile){
$loopy = "0";
exit();
}

}
}
}

Sunday, April 4, 2010

Installing all new crap on debian / ubuntu

THis is the same as the script before as written for both arch and ubuntu...the diffrence being is that if you aren't root you need to run this with sudo. Just remember save the script with a name like install.pl and launch it from the command line with a cammand like: sudo perl install.pl

The script:


#!/usr/bin/perl
use strict;
use warnings;

my $aptget;

my @aptget_installs = qw(
xchat screen cpan
wget pcre-devel cmake
gtk+ pygame python-devel
cmake cmake-devel liblo
liblo-devel gcc-c++-devel cmake-gui
libsndfile libsndfile-devel qjackctl
zynaddsubfx vkeybd qtjack-devel
csound csound-devel portmidi
rosegarden4 jamin jack-audio-connection-kit-devel
ctapi-cyberjack-devel zynjacku ardour
nxt_python pybluez
);

&array_cracker();

sub array_cracker(){
foreach(@aptget_installs){
print $_;
$aptget = `apt-get install -y $_`;
print $aptget;
}
}

Saturday, April 3, 2010

Installing a bunch of crap on arch linux through a perl script.

Rewrote the script I made for setting up my music and dev stuff for fedora, in perl:

#!/usr/bin/perl
use strict;
use warnings;

my $pacman;

my @pacman_installs = qw(
xchat screen cpan
wget pcre-devel cmake
gtk+ pygame python-devel
cmake cmake-devel liblo
liblo-devel gcc-c++-devel cmake-gui
libsndfile libsndfile-devel qjackctl
zynaddsubfx vkeybd qtjack-devel
csound csound-devel portmidi
rosegarden4 jamin jack-audio-connection-kit-devel
ctapi-cyberjack-devel zynjacku ardour
nxt_python pybluez
);

&array_cracker();

sub array_cracker(){
foreach(@pacman_installs){
print $_;
$pacman = `pacman -Syq --noconfirm $_`;
print $pacman;
}
}

Friday, April 2, 2010

Rsync slowness?

I was having some slow rsync issues on my all gig ethernet network. I was rsyncing as such and was having less than stellar download speeds. My network is setup with one macbook, a linksys routers[gig ethernet](one of the slightly higher end models not the standard wrt54g). And it's connected on my fedora box, here are the speeds I was getting:

My "rsync -avz -P /Users/lane/Downloads/ root@192.168.1.201:/home/lane/Downloads/rsync/":
Code:
Bones/Bones Season 2/Bones.S02E05.HDTV.XviD.avi
366486588 100% 7.42MB/s 0:00:47 (xfer#11, to-check=858/1014)
Bones/Bones Season 2/Bones.S02E06.HDTV.XviD.avi
365428736 100% 7.41MB/s 0:00:47 (xfer#12, to-check=857/1014)
Bones/Bones Season 2/Bones.S02E07.HDTV.XviD.avi
366942838 100% 6.59MB/s 0:00:53 (xfer#13, to-check=856/1014)
Bones/Bones Season 2/Bones.S02E08.HDTV.XviD.avi
366863346 100% 7.25MB/s 0:00:48 (xfer#14, to-check=855/1014)
Bones/Bones Season 2/Bones.S02E09.HDTV.XviD.avi
366988952 100% 7.08MB/s 0:00:49 (xfer#15, to-check=854/1014)
Bones/Bones Season 2/Bones.S02E10.HDTV.XviD.avi
366969326 100% 6.37MB/s 0:00:54 (xfer#16, to-check=853/1014)
Bones/Bones Season 2/Bones.S02E11.HDTV.XviD.avi
366988976 100% 7.19MB/s 0:00:48 (xfer#17, to-check=852/1014)
Bones/Bones Season 2/Bones.S02E12.HDTV.XviD.avi
366975374 100% 7.60MB/s 0:00:46 (xfer#18, to-check=851/1014)
Bones/Bones Season 2/Bones.S02E13.HDTV.XviD.avi
367679488 100% 6.76MB/s 0:00:51 (xfer#19, to-check=850/1014)
Bones/Bones Season 2/Bones.S02E14.HDTV.XviD.avi
367340574 100% 6.69MB/s 0:00:52 (xfer#20, to-check=849/1014)
Bones/Bones Season 2/Bones.S02E15.HDTV.XviD.avi
366983676 100% 6.98MB/s 0:00:50 (xfer#21, to-check=848/1014)
Bones/Bones Season 2/Bones.S02E16.HDTV.XviD.avi
366794752 100% 7.34MB/s 0:00:47


I ran this command on my linux box
ethtool -s eth0 speed 1000 duplex full
I think ifconfig eth0 down then ifconfig eth0 up and restarted the rsync.
And the speeds increased but not dramatically:

ue_Gender1-26(complete)/zx.blue.gender.05.divx503.ogm
237303388 100% 17.56MB/s 0:00:12 (xfer#1, to-check=887/1014)
Blue_Gender1-26(complete)/zx.blue.gender.06.divx503.ogm
238456995 100% 11.51MB/s 0:00:19 (xfer#2, to-check=886/1014)
Blue_Gender1-26(complete)/zx.blue.gender.07.divx503.ogm
237551102 100% 5.34MB/s 0:00:42 (xfer#3, to-check=885/1014)
Blue_Gender1-26(complete)/zx.blue.gender.11.divx503.ogm
243860297 100% 16.20MB/s 0:00:14 (xfer#4, to-check=881/1014)
Blue_Gender1-26(complete)/zx.blue.gender.12.divx503.ogm
242815771 100% 16.70MB/s 0:00:13 (xfer#5, to-check=880/1014)
Blue_Gender1-26(complete)/zx.blue.gender.13.divx503.ogm
239553347 100% 20.09MB/s 0:00:11 (xfer#6, to-check=879/1014)
Blue_Gender1-26(complete)/zx.blue.gender.14.divx503.ogm
237805952 100% 11.54MB/s 0:00:19 (xfer#7, to-check=878/1014)
Blue_Gender1-26(complete)/zx.blue.gender.15.divx503.ogm
237405118 100% 16.57MB/s 0:00:13 (xfer#8, to-check=877/1014)
Blue_Gender1-26(complete)/zx.blue.gender.16.divx503.ogm
238030584 100% 17.48MB/s 0:00:12 (xfer#9, to-check=876/1014)
Blue_Gender1-26(complete)/zx.blue.gender.20.divx503.ogm

However, this still isn't satisfactory. After some more investigation I started using the mii-tool
mii-tool -v
Which showed that my eth0 card was still at the same speed:
eth0: negotiated 100baseTx-FD flow-control, link ok

This does not explain the speed increase if this tool is accurate.
I decided to then investigate the mac and after a greping of the dmesg found this:
sudo dmesg |grep 'en0'
Ethernet [AppleYukon2]: Link up on en0, 1-Gigabit, Full-duplex, Symmetric flow-control, Debug [796d,af08,0de1,0200,cde1,2800]

And while it is not definite proof that the port is running at gigabit it does make me want to look to other areas for a speed boost. Does rsync just have an incredible amount of overhead? Or is there something else wrong? And if something else is wrong, is it the router, or one of the boxes...perhaps I should just be less picky about my speeds. If you have any answers let me know.