To perform quick Windows installations or reinstalls, if you have .wim images, you may use the ImageX application.
You'll firstly need to copy the ImageX files on a bootable USB drive - or whatever you prefer - and the .wim images on a support accessible from your PC at boot time (network drive, system partition, USB disk, etc...)
Now reboot the system and configure the bootloader to load the ImageX support first; you'll be brought to a command line shell you can interact with.
Note: If you need to alter the disk partitioning, run the diskpart command first.
To install the image(s) simply run the following command:
PATH_TO_IMAGEX_DRIVE\imagex.exe /apply PATH_TO_IMAGES\IMAGE_NAME WIM_VOLUME_NUMBER DRIVE_LETTER
eg:
G:\imagex.exe /apply H:\images\win7.wim 1 C:\
Copying the image onto the specified destination may take some time, so just sit back and wait. Here's a full list of ImageX's command line options.
24/12/2013
[Windows] Create bootable USB drive
To make a USB disk bootable on Windows, you can use the diskpart application. From an administrator command line, run diskpart then type the following commands:
- list disk - to find which one is the USB flash drive you want to make bootable
- select disk X - where X is the USB drive disk number you found from the previous command
- clean - to delete all data from it
- create part pri - to create a new primary partition
- select part 1 - to select the newly created partition
- format fs=Y quick - where Y is either ntfs or fat32 to perform a quick format to the desired filesystem
- active - to render said partition active
- exit
Now copy whatever you need in the root of the USB stick and you're set.
21/12/2013
[iPhone] Manage contacts and import/export vCards
iTunes may have some good syncing capabilities, but the contact management is abysmal; for example, the very basic vCard import/export functionality is missing.
Luckily, there's a third-party app for that: Wondershare TunesGo. Just install, connect to your device and select the contact menu to start managing them.
Luckily, there's a third-party app for that: Wondershare TunesGo. Just install, connect to your device and select the contact menu to start managing them.
[iPhone] Set custom ringtone with longer duration than Apple's limit
iTunes allows you to upload custom ringtones to your iPhone only if their duration is less than 30 seconds. An easy way to circumvent that limitation, without jailbreaking the device, is to trick iTunes itself.
Begin with following the standard procedure to add a valid song (any .m4r shorter than the duration limit) to your Tones library, but be sure to name it exactly as the ringtone you really want to add. Follow all the steps but stop BEFORE syncing with the device.
Now create a .m4r version of the ringtone you actually want (from iTunes, create AAC version, then manually rename it to .m4r) and save it somewhere. From the filesystem explorer, browse to the folder containing the short ringtone and rename it .bak (or whatever), then copy the full length version in that same folder; make sure that the two files are named the same way, except for the .bak extension.
Finally, you can start syncing. iTunes will load the full length ringtone instead of the short one, and will not complain about it.
Begin with following the standard procedure to add a valid song (any .m4r shorter than the duration limit) to your Tones library, but be sure to name it exactly as the ringtone you really want to add. Follow all the steps but stop BEFORE syncing with the device.
Now create a .m4r version of the ringtone you actually want (from iTunes, create AAC version, then manually rename it to .m4r) and save it somewhere. From the filesystem explorer, browse to the folder containing the short ringtone and rename it .bak (or whatever), then copy the full length version in that same folder; make sure that the two files are named the same way, except for the .bak extension.
Finally, you can start syncing. iTunes will load the full length ringtone instead of the short one, and will not complain about it.
17/12/2013
[Java] Compute discrete logarithm
The discrete logarithm is a widely used concept in cryptography since it's pretty easy and fast to compute when knowing the crypto secrets, but it's otherwise really hard to crack by normal means.
In this example, from Coursera's cryptography I course, we'll try to compute it for some 153-digits numbers using a MITM technique. The exercise is as such:
In this example, from Coursera's cryptography I course, we'll try to compute it for some 153-digits numbers using a MITM technique. The exercise is as such:
Your goal this week is to write a program to compute discrete log modulo a prime p. Let g be some element in Z*p and suppose you are given h in Z*p such that h = g^x where x is between 1 and 2^40. A basic brute force attack is to just try all possible values in 2^40 time, but our MITM attack will take just 2^20 time.
Let B=2^20. Since x is less than B^2 we can write the unknown x base B as x=x0 B+x1 where x0,x1 are in the range [0,B-1]. Then h = g^x = g^(x0 B+x1) = (g^B)^x0·g^x1 in Zp. And then we have h/(g^x1) = (g^B) ^ x0 in Zp.
The variables in this equation are x0,x1 and everything else is known: you are given g,h and B=2^20. Since the variables x0 and x1 are now on different sides of the equation we can find a solution using meet in the middle:
- First build a hash table of all possible values of the left hand side h/(g^x1) for x1=0,1,…,2^20.
- Then for each value x0=0,1,2,…,2^20 check if the right hand side (g^B)^x0 is in this hash table. If so, then you have found a solution (x0,x1) from which you can compute the required x as x=x0B+x1.
We will now find x such that h = g^x.
[Java] Hashing with SHA-256
As for the cryptography algorithms, Java offers native libraries for hashing too.
In this example I'll describe how to use apply the SHA-256 hashing to a file. However, following the example found in Coursera's cryptography I course, we won't compute the whole file's hash at once; instead, we'll break the file into 1KB blocks (1024 bytes). Then we'll compute the hash of the last block and appended said value to the second to last block. Then again we'll compute the hash of this augmented second to last block and the resulting hash is appended to the third block from the end, and so on..
Update: Hashing this way doesn't make the hash stronger or weaker, but allows us to stream the file while hashing instead of having to wait for the whole operation to complete.
Update: Hashing this way doesn't make the hash stronger or weaker, but allows us to stream the file while hashing instead of having to wait for the whole operation to complete.
As usual, auxiliary functions such as string2Hex can be found here.
The basic imports are:
Also note, we're using Apache commons library to handle our File operations
import java.io.ByteArrayInputStream;
import java.io.DataInputStream;
import java.io.File;
import java.security.MessageDigest;
import java.util.LinkedList;
import java.util.List;
import javax.xml.bind.DatatypeConverter;
import org.apache.commons.io.FileUtils;
Also note, we're using Apache commons library to handle our File operations
[Java] AES encryption and decryption
To perform AES encryption and decryption, without implementing your own algorithms - since it is usually unsafe - you may use the standard Java crypto libraries. A more complete crypto reference is found in the JCA reference guide.
30/11/2013
Access NAT and Network configuration tool in VmWare Player
The NAT and Network configuration tool available for Windows users in VmWare Workstation used to be available for VmWare Player users too but since release 6, it's become a little harder to get.
All you need are two files: vmnetcfg.exe and vmnetcfglib.dll. You can find them both in a typical Workstation installer (the free trial version too), by running this command against the installer:
VMWare-worksation-INSTALLER_NAME.exe /e .\unpack
now just browse to the newly created unpack folder and rename _vmnetcfglib.dll without the leading underscore, then copy both in your Player installation directory.
22/11/2013
[Oracle] 12c install error INS-30131
While trying to install Oracle 12c on Windows, in the very first steps, you may encounter an INS 30131 error saying:
[INS-30131] Initial setup required for the execution of installer validations failed. Cause: Failed to access the temporary location
This may happen because upon installation, Oracle's trying to access the C drive as a network share but fails to do so. To fix it, make sure you can actually access the C$ share on your machine.
[INS-30131] Initial setup required for the execution of installer validations failed. Cause: Failed to access the temporary location
This may happen because upon installation, Oracle's trying to access the C drive as a network share but fails to do so. To fix it, make sure you can actually access the C$ share on your machine.
[Windows] Cannot access hidden C admin share
If you get an error while trying to access the Windows hidden C share (C$):
net use \\localhost\c$
System error 53 has occurred.
The network path was not found.
A simple thing is just to make sure your TCP/IP NetBIOS Helper and Server services are running (Start-Run, services.msc) and try again:
net use \\localhost\c$
The command completed successfully.
Of course, your user must be either an administrator or be part of the administrator group.
If it still fails, manually edit the registry (Start-Run, regedit). Browse to:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System
and create a new DWORD value LocalAccountTokenFilterPolicy set to 1
If it still fails, manually edit the registry (Start-Run, regedit). Browse to:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System
and create a new DWORD value LocalAccountTokenFilterPolicy set to 1
08/11/2013
[Java] Hexadecimal to Binary
To convert a String representation of an hex to its binary String representation in Java:
static String[]hex={"0","1","2","3","4","5","6","7","8","9","A","B","C","D","E","F"};
static String[]binary={"0000","0001","0010","0011","0100","0101","0110","0111","1000","1001","1010","1011","1100","1101","1110","1111"};
public static String hex2Binary(String hexStr){
String bin="";
for(int i=0;i<hexStr.length();i++){
String temp = String.valueOf(hexStr.charAt(i));
for(int j=0;j<hex.length;j++){
if(temp.equalsIgnoreCase(hex[j]))bin+=binary[j];
}
}
return bin;
}
[Java] String to Hexadecimal and viceversa
To convert a String to its hex representation in Java, simply:
and in the other direction:
public static String string2Hex(String str){
char[] chars = str.toCharArray();
StringBuffer hex = new StringBuffer();
for(int i = 0; i < chars.length; i++){
hex.append(Integer.toHexString((int)chars[i]));
}
return hex.toString();
}
and in the other direction:
public static String hex2String(String hex){
StringBuilder sb = new StringBuilder();
/*two hex characters for each ASCII one*/
for( int i=0; i<hex.length()-1; i+=2 ){
String output = hex.substring(i, (i + 2));
int decimal = Integer.parseInt(output, 16);
sb.append((char)decimal);
}
return sb.toString();
}
28/10/2013
[Java 4] Download file from portlet v1.0
Same scenario as last time, but a cleaner solution: link a servlet directly from a JSP page and have the server serve the file as a response instead of a new JSP.
[Java] BigInteger square root
Recently I was following a cryptography course and one assignment required us to compute square roots of very huge numbers. In Java, you require a BigInteger object to correctly represent numbers bigger than 231-1 (Integer max size).
Those BigInteger objects come with the basic operations implemented (addition, subtraction, multiplication and division) plus some other useful ones (exponentiation and modulus for example) but lack a method to compute the square root and it's highly unlikely that it will be added in the future[citation needed]. The only solution is either to implement your own method or find a reliable library.
21/10/2013
Kill Oracle RAC session from Toad
Toad versions previous than 11 accessing an Oracle database version prior to 11g, do not support killing sessions directly from the session browser, if the database being accessed is configured as a cluster.
As a workaround, aside from directly connecting to the desired instance, you may add to the tnsnames.ora file the TNS of the particular DB in the RAC from which you need to kill a session, then reload Toad.
As a workaround, aside from directly connecting to the desired instance, you may add to the tnsnames.ora file the TNS of the particular DB in the RAC from which you need to kill a session, then reload Toad.
[PL/SQL] Oracle continue loop with goto
Oracle database versions previous than 11g do not support the CONTINUE statement to skip to the end of a loop and keep cycling. As an alternative, you can use the GOTO statement WITH CAUTION.
Firstly, declare a label which marks the point were you want the execution to continue from using:
Firstly, declare a label which marks the point were you want the execution to continue from using:
<<label_name>>
and remember that the label must be followed by an executable statement; in practice, it means you cannot place it directly before the END LOOP instruction. Instead, follow it with a NULL statement as in:
<<label_name>>
NULL;
END LOOP;
Then simply skip to it as needed with:
GOTO label_name;
12/10/2013
[Java] Keep HTML form filled after failed Spring validation
I was working on a JSR-168 portlet written in Java 4 using Spring 2 and I had plain HTML forms validated via Spring validators. Everything works fine with this setting but upon failed validation, the page would reload showing the errors and an empty form.
Instead of replacing all the forms with Spring ones (form:form), I found an alternative solution which let me keep my current project structure and tone down the necessary modifications.
08/10/2013
[Java] Store session attributes in portlet controller and retrieve them from JSP
I was coding a portlet following the JSR-168 specification and Spring 2 and I noticed an inconsistent behaviour on some pages. Passing some Map objects, needed to initialize the JSP drop-down menus, by storing them in session in the controller and retrieving them in the JSP, would work always except when the page was reloaded after a failed form validation during the submission phase.
Note: this issue could not be solved by simply using the ModelAndView addObject("name", value) method in the controller and then retrieving the object in the JSP with ${name} outside of a Java block, since I needed to perform some actions on those values directly from the JSP, therefore inside a Java block, without using EL.
25/09/2013
[Java] Spring use DAO in JSP page
So you're happily coding your webapp with Spring, and have all your DAOs correctly defined as beans; you know they work because you are constantly injecting them around, but now you need to access one of them from a JSP.. how?
You may find this code fragment useful:
You import your DAO, and request it directly from the JSP. Note that this example relies on the fact that somewhere you have correctly defined a bean named "myDao".
You may find this code fragment useful:
<%@page import="com.groglogs.mydaos.MyDao"%>
<%@page import="org.springframework.web.context.WebApplicationContext"%>
<%@page import="org.springframework.web.context.support.WebApplicationContextUtils"%>
<%
MyDao myDao = null;
WebApplicationContext context = WebApplicationContextUtils.getWebApplicationContext(getServletContext());
if (context != null) {
myDao = (MyDao) context.getBean("myDao");
}
%>
You import your DAO, and request it directly from the JSP. Note that this example relies on the fact that somewhere you have correctly defined a bean named "myDao".
19/09/2013
[Eclipse] Comparison method violates its general contract error when updating
When installing updates in Eclipse, you may encounter the "Comparison method violates its general contract" error.
This may be related to a known bug in older Eclipse versions when using new JDKs (>6), I encountered it even using a JDK6 with latest Helios though. To fix it, try adding:
-Djava.util.Arrays.useLegacyMergeSort=true
in your eclipse.ini file, after the -vmargs flag, then restart Eclipse and run the update manager again.
This may be related to a known bug in older Eclipse versions when using new JDKs (>6), I encountered it even using a JDK6 with latest Helios though. To fix it, try adding:
-Djava.util.Arrays.useLegacyMergeSort=true
in your eclipse.ini file, after the -vmargs flag, then restart Eclipse and run the update manager again.
[Eclipse] Slow update process
When installing updates via the Eclipse update manager, it sometimes takes forever to complete the update process, even for very small updates.
One cause may be Eclipse trying to contact all known sites when fetching updates. To avoid that, try adding:
-Declipse.p2.mirrors=false
in your eclipse.ini file, after the -vmargs flag then restart Eclipse. Alternatively, you can uncheck the "Contact all update sites" flag every time you run an update from the update wizard.
One cause may be Eclipse trying to contact all known sites when fetching updates. To avoid that, try adding:
-Declipse.p2.mirrors=false
in your eclipse.ini file, after the -vmargs flag then restart Eclipse. Alternatively, you can uncheck the "Contact all update sites" flag every time you run an update from the update wizard.
10/09/2013
[Windows] Manage printers from command line
On Windows, you can manage the printers connected to your computer from the command line either via Rundll32 printui.dll,PrintUIEntry on 32-bit systems or Cscript.exe and the Prnmngr.vbs script on 64-bit systems.
23/08/2013
[Android] Auto Answer app
Last year I bought an Android smartphone from LG, with Gingerbread 2.3.4 on it. Much to my surprise, it was not very smart, for example I couldn't set it to automatically answer incoming calls.
The good thing about having it run Android, was that I could fix this flaw by developing my own app, so I googled around and found an Auto Answer project on Google Code from Matt Hahnfeld, released under a GNU GPL v3 licence.
Since the author removed it from Google's Play Store, I catered his code to my needs and installed the app on my Optimus HUB. It works perfectly, I tested it on Ice Cream Sandwich 4.0.3 (LG Optimus L7) without issues, and it should work on Froyo 2.2 too.
At the time there weren't many free apps that did this but now you finally have some choices, note that some phones already come with this feature built-in.
You can find my code on GitHub, the licence remains the same GNU GPL v3. To successfully compile it you'll need an IDE such as Eclipse, and the proper Android SDK for your OS version. The source code I uploaded comes with the Android 4 lib, but you can easily switch it to another version.
The good thing about having it run Android, was that I could fix this flaw by developing my own app, so I googled around and found an Auto Answer project on Google Code from Matt Hahnfeld, released under a GNU GPL v3 licence.
Since the author removed it from Google's Play Store, I catered his code to my needs and installed the app on my Optimus HUB. It works perfectly, I tested it on Ice Cream Sandwich 4.0.3 (LG Optimus L7) without issues, and it should work on Froyo 2.2 too.
At the time there weren't many free apps that did this but now you finally have some choices, note that some phones already come with this feature built-in.
You can find my code on GitHub, the licence remains the same GNU GPL v3. To successfully compile it you'll need an IDE such as Eclipse, and the proper Android SDK for your OS version. The source code I uploaded comes with the Android 4 lib, but you can easily switch it to another version.
22/08/2013
[Java] String to Date and vice versa
When working with dates in Java you'll often find yourself having to convert them from a String object to a Date object and vice versa. These function snippets will allow you to easily perform the conversion operations, no third-party libraries needed:
Imports
import java.text.Format;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.Locale;
string2Date
date2String
Of course, if you prefer, rather than returning NULL if something went wrong, you could always raise an Exception. You can find all accepted formats in the SimpleDateFormat JavaDoc.
Imports
import java.text.Format;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.Locale;
string2Date
/** Converts given String in a Date object with specific format
* @param date - a String representing the date to convert. IT MUST BE in the given format
* @param dateFormat - a String specifying the format of the date e.g: dd/MM/yyyy
* @return a Date object, {@null} if the given string is {@null} or if there's been a conversion error
* */
public static Date string2Date(String date, String dateFormat){
Date d = null;
try{
if(date!=null && !date.equalsIgnoreCase("") && dateFormat!=null && !dateFormat.equalsIgnoreCase("")) d = new SimpleDateFormat(dateFormat, Locale.ITALIAN).parse(date);//or whatever Locale you need
}catch(Exception e){}//you could always decide to raise the exception rather than returning simply NULL
return d;
}
date2String
/** Converts given Date in a String object with specific format
* @param date - a Date to be represented as a string. IT MUST BE in the given format
* @param dateFormat - a String specifying the format of the date e.g: dd/MM/yyyy
* @return a String object, {@null} if the given date is {@null} or if there's been a conversion error
* */
public static String date2String(Date date, String dateFormat){
Format formatter = new SimpleDateFormat(dateFormat);
String s = null;
try{
if(date!=null && dateFormat!=null && !dateFormat.equalsIgnoreCase(""))s = formatter.format(date);
}catch(Exception e){}//you could always decide to raise the exception rather than returning simply NULL
return s;
}
Of course, if you prefer, rather than returning NULL if something went wrong, you could always raise an Exception. You can find all accepted formats in the SimpleDateFormat JavaDoc.
17/08/2013
28/07/2013
[Java 4] Download file from portlet
Scenario: Java 4, Spring 2, WebSphere 6, JSR-168. How the hell do I let the user download a file from a portlet?
The file is extracted from a database and does not exist on the server's filesystem yet; redirecting to a servlet to create and send the response which contains the file to download fails with a generic:
java.lang.IllegalStateException: Can't invoke sendRedirect() after certain methods have been called
More specifically, that method can not be invoked after any of the following methods of the ActionResponse interface has been called:
The file is extracted from a database and does not exist on the server's filesystem yet; redirecting to a servlet to create and send the response which contains the file to download fails with a generic:
java.lang.IllegalStateException: Can't invoke sendRedirect() after certain methods have been called
More specifically, that method can not be invoked after any of the following methods of the ActionResponse interface has been called:
- setPortletMode
- setWindowState
- setRenderParameter
- setRenderParameters
- removePublicRenderParamter
[Java] Get current user from portlet
To get the user currently authenticated on the portal from a portlet, you may use:
Principal p = request.getUserPrincipal();
you may then call the getName method to get the UID of the user. If the user is not authenticated or the page is not protected, the Principal object is null.
Principal p = request.getUserPrincipal();
you may then call the getName method to get the UID of the user. If the user is not authenticated or the page is not protected, the Principal object is null.
[Java] Get extension from filename
To extract the extension from a given filename in Java, you may try:
This works even if filename contains multiple dots, however it's not an accurate way of determining a file type.
String yourfilename = "something.ext";
String[] tokens = yourfilename.split("\\.(?=[^\\.]+$)");
String extension = tokens[1].toLowerCase();
This works even if filename contains multiple dots, however it's not an accurate way of determining a file type.
[Java] Send mail
Here's an example on how to send HTML emails to multiple recipients, without attachments, via Java.
This Mailer class was written in Java 4 and relies upon the javax.mail.* libraries, providing a method send which takes 4 parameters:
This Mailer class was written in Java 4 and relies upon the javax.mail.* libraries, providing a method send which takes 4 parameters:
- List
:rcptTo - "TO" recipients - List
:ccs - "CC" recipients - String:subject - message subject
- String:body - message body
13/07/2013
[Java 4] Replace placeholder word in string
Should you be unlucky and have to work with Java 4, you may find it lacking many of the convenient features found in later versions.
One of them is the ability to replace a placeholder word in a String with little effort. What you can do is use the replaceAll method:
String str = "some string with a #placeholder# inside";
str=str.replaceAll("#placeholder#", "value");
Note that it will search for and replace ALL matches it finds, making it viable as a placeholder substitution method while it's pretty useless if you have to replace a single word; in that case, you may find the replaceFirst method more suitable.
Remember in any case that the first parameter is parsed as a regular expression meaning that you'll have to escape reserved characters such as "$" if your placeholder contains them. Another very important thing to remember is that, since String objects are immutable in Java, if you don't assign the return value to some variable, the substitution won't have any effect.
01/07/2013
[Java] Convert InputStream to String
This code snippet is the work of Pavel Repin from StackOverflow, and it's highly useful if work in Java. It allows you to easily convert an InputStream to a String taking advantage of the Scanner class, plus, since it's a native Java class, you don't need to download additional jars:
public static String convertStreamToString(java.io.InputStream is) {
java.util.Scanner s = new java.util.Scanner(is).useDelimiter("\\A");
return s.hasNext() ? s.next() : "";
}
[Java] Print document programmatically
Should you need to print a document from your Java application, you may find this code snippet, from the Java AWT Desktop class, useful:
import java.awt.Desktop;
import java.io.File;
import java.io.IOException;
public class Main{
public static void main (String [] args){
try {
File f = new File("PATH_TO_FILE_TO_PRINT");
Desktop.getDesktop().print(f);
} catch (IOException e) {
// Handle the exception here if needed
e.printStackTrace();
}
}
}
note however, that the command runs on the same machine where the JVM is running! If the application you're accessing runs on a separate host machine, it will not work at all; instead you may think of creating an applet to run this code on the client machine, passing the file to print as a parameter (remote paths are fine too!)
28/06/2013
[Python] Map/Reduce examples
These are simple algorithms written following the Map/Reduce model in Python. To successfully run them, you must download this archive containing:
- MapReduce.py and MapReduce.pyc: a simple implementation of the Map/Reduce programming model in Python
- books.json, dna.json, friends.json, matrix.json and records.json: simple datasets
[SQL] Matrix multiplication
The second assignment in my Coursera Big Data course regarded data analysis in SQL using SQLite3. We used a test dataset, which you can download from here, composed of:
- matrix.db: a simple database representing two square sparse matrices stored as tables A and B each with row_num, col_num and value as columns
- reuters.db: a database containing a single table frequency(docid, term, count), where docid is an identifier corresponding to a particular file, term is an English word, and count is the number of the times the term appears within the document indicated by docid
I will not post all scripts produced here since many were just simple SELECTs so we'll just focus on the most interesting stuff: matrix multiplication in SQL.
It sounds more complicated than it really is but, given the two sparse matrices, it is possible to compute their multiplication AxB by doing a simple JOIN between A columns and B rows, then GROUPing BY A rows and B columns and finally SELECTing, for each matrix cell, the SUM of the multiplication between the two cell values in both matrices:
SELECT a.row_num, b.col_num, SUM(a.value*b.value)
FROM A a JOIN B b ON a.col_num=b.row_num
GROUP BY a.row_num, b.col_num;
Which is exactly the implementation of the matrix multiplication formula.
FROM A a JOIN B b ON a.col_num=b.row_num
GROUP BY a.row_num, b.col_num;
Which is exactly the implementation of the matrix multiplication formula.
[Python] Twitter tweets analysis
I recently took a Big Data course on Coursera, where I had the chance to employ different analysis techniques to huge datasets. One of the assignments required us to perform some sort of tweet sentiment analysis using code written in Python (version 2.7.3).
To successfully run the examples, you must download this archive which contains:
- README.html: instructions on how to register on Twitter and get your own token and consumer keys
- AFINN-README.txt: a description of the AFINN-111.txt file which will be used to evaluate the various tweets
- AFINN-111.txt: a list of english terms and their sentiment score, manually set by Finn Ã…rup Nielsen
- twitterstream.py: a Python script used to query Twitter and collect a stream of tweets. It runs indefinitely so you will have to manually stop it after some time
Firstly, you should run twitterstream.py as:
python twitterstream.py > output.txt
stopping it when you feel you've collected enough data. It will extract some tweets in JSON format and store them in a new file called output.txt which will be our test dataset. Note that all scripts were automatically graded so many fine tuning techniques had to be avoided to prevent it failing to correctly evaluate each solution.
Lastly, to understand the scripts, you should take a look at Twitter's official tweets JSON format documentation.
19/06/2013
[PL/SQL] Read, parse and store data from an XML file
Suppose you have a fairly complex XML file with this structure:
<myRoot>
<myElement>
<aNode>
<aData>123</aData>
<bData><![CDATA[some lengthy text with special <characters>in</it>]]></bData>
</aNode>
<otherNode>
<otherData>31/12/1900</otherData>
.
.
.
<otherData>01/01/2000</otherData>
</otherNode>
<lastNode>
<simpleTag>
<simpleValue1>some text</simpleValue1>
<simpleValue2>other text</simpleValue2>
<simpleFilter>0</simpleFilter>
</simpleTag>
<simpleTag>
<simpleValue1>blah blah</simpleValue1>
<simpleValue2>other blah</simpleValue2>
<simpleFilter>1</simpleFilter>
</simpleTag>
<simpleTag>
<simpleValue1>foo</simpleValue1>
<simpleValue2>bar</simpleValue2>
<simpleFilter>1</simpleFilter>
</simpleTag>
.
.
.
</lastNode>
</myElement>
.
.
.
</myRoot>
Also, we won't have a proper schema file but we know that it'll have one or more myElement nodes, for each of which it'll have:
- a single aNode node with an integer aData element and a CDATA bData text element
- a single otherNode node with multiple otherData elements representing dates
- a single lastNode node with multiple simpleTag child nodes, each composed of three elements: a string in both simpleValue1 and simpleValue2 and an integer in simpleFilter. Additionally, we also know that for every lastNode node, just one and only one simpleTag node will have its simpleFilter element set to 0.
Now you want to store it in a table, then read it, parse its contents and save them in another table.
15/06/2013
[PL/SQL] Oracle send mail through SMTP server with authentication
Sending mail through a PL/SQL script in Oracle is pretty easy thanks to the UTL_SMTP package.
Much like TELNET, sending a mail via our script will require us to use the HELO, MAIL FROM, RCPT TO and DATA commands. Thus, when sending a mail to multiple recipients - directly or in CC - we must issue the RCPT TO command once for each one and add the CC mail header to our message in order for them to appear as such.
Much like TELNET, sending a mail via our script will require us to use the HELO, MAIL FROM, RCPT TO and DATA commands. Thus, when sending a mail to multiple recipients - directly or in CC - we must issue the RCPT TO command once for each one and add the CC mail header to our message in order for them to appear as such.
31/05/2013
[PL/SQL] Create and call Java procedure to read directory content
Sometimes even PL/SQL isn't enough; luckily Oracle gives us the ability to store Java functions and treat them as procedures.
Consider the following example: we know some files will be stored in a certain directory without knowing their names beforehand and we'd like to read and process those files from our PL/SQL code. By creating a simple Java function depending on the File class, we can solve our problem quickly and easily.
Tag:
HowTo,
Java,
Oracle,
PL/SQL,
Source code
11/05/2013
Oracle 11g regular expressions cheatsheet
General Information | |||||||
Anchoring Characters |
|
22/03/2013
Oracle update table while looping on it
You may find yourself needing to update a table while looping on it:
DECLARE
newValue someType;
BEGIN
FOR myCur IN (SELECT key FROM table)LOOP
newValue:=myFunction(key);
UPDATE table
SET column=newValue
WHERE key=myCur.key;
COMMIT;
END LOOP;
END;
/
This however is not possible in Oracle, instead you should fetch the table keys into an array before and then cycle it:
DECLARE
newValue someType;
TYPE vArrayType IS VARRAY([size]) OF someType; --use VARRAY if you already know the size!
myArray vArrayType;
BEGIN
--fill the array
SELECT key
BULK COLLECT INTO myArray
FROM table;
--loop on it
FOR i IN myArray.FIRST..myArray.LAST LOOP
newValue:=myFunction(myArray(i));
UPDATE table
SET column=newValue
WHERE key=myArray(i);
COMMIT;
END LOOP;
END;
/
DECLARE
newValue someType;
BEGIN
FOR myCur IN (SELECT key FROM table)LOOP
newValue:=myFunction(key);
UPDATE table
SET column=newValue
WHERE key=myCur.key;
COMMIT;
END LOOP;
END;
/
This however is not possible in Oracle, instead you should fetch the table keys into an array before and then cycle it:
DECLARE
newValue someType;
TYPE vArrayType IS VARRAY([size]) OF someType; --use VARRAY if you already know the size!
myArray vArrayType;
BEGIN
--fill the array
SELECT key
BULK COLLECT INTO myArray
FROM table;
--loop on it
FOR i IN myArray.FIRST..myArray.LAST LOOP
newValue:=myFunction(myArray(i));
UPDATE table
SET column=newValue
WHERE key=myArray(i);
COMMIT;
END LOOP;
END;
/
06/03/2013
Oracle error ORA-04091 table is mutating, trigger/function may not see it
When working with triggers in Oracle, you may encounter the "ORA-04091 table is mutating, trigger/function may not see it" error. This may happen when you have a trigger running on a table for an insert, update or delete operation and you're running a function which accesses a table being modified at the same time as the trigger.
The fact is, you may not perform further operations on a table which is currently being INSERTed into, UPDATEd or DELETEd FROM in a single statement. For example, you have a BEFORE trigger on myTable and try to run this statement:
UPDATE myTable
SET myColumn = myFunction(someValue);
It will result in our error if myFunction works on myTable to produce a result; this other statement however will not raise an error as the function and trigger executions will be kept separated:
myVariable := myFunction(someValue);
UPDATE myTable
SET myColumn = myVariable;
The fact is, you may not perform further operations on a table which is currently being INSERTed into, UPDATEd or DELETEd FROM in a single statement. For example, you have a BEFORE trigger on myTable and try to run this statement:
UPDATE myTable
SET myColumn = myFunction(someValue);
It will result in our error if myFunction works on myTable to produce a result; this other statement however will not raise an error as the function and trigger executions will be kept separated:
myVariable := myFunction(someValue);
UPDATE myTable
SET myColumn = myVariable;
05/03/2013
Windows delete file type association
Some extensions in Windows are automatically registered and assigned to be opened by a specific application, others can be manually set via the Open with... dialog or Control panel->Default programs. In either case however, you cannot delete an association.
You may do so by directly editing the registry (Start->Run->regedit):
Assuming you'd like to remove the association for the EXT extension, browse to HKEY_CURRENT_USER\Software\ Microsoft\Windows\ CurrentVersion\Explorer\ FileExts\
And delete the subtree named .EXT (with the dot before). Now browse to HKEY_CLASSES_ROOT\ and delete the subtree named .EXT (again, with the dot before) and EXT_auto_file
You may do so by directly editing the registry (Start->Run->regedit):
Assuming you'd like to remove the association for the EXT extension, browse to HKEY_CURRENT_USER\Software\
And delete the subtree named .EXT (with the dot before). Now browse to HKEY_CLASSES_ROOT\ and delete the subtree named .EXT (again, with the dot before) and EXT_auto_file
Oracle SELECT TOP
When working with Oracle, you may need to select the TOP row (or top N rows) from a query result, but you'll find there is no such statement.
While you can certainly use ROWNUM, you should note that it is executed BEFORE any ORDER BY statement, thus producing wrong results in that case. The solution in this case is as simple as:
SELECT *
FROM (SELECT *
FROM myTable
ORDER BY myColumn)
WHERE ROWNUM = 1;
By encapsulating your query with the ORDER BY clause inside another query using ROWNUM, you now effectively order the dataset before extracting the TOP row. You may obviously use it as TOP N by setting another value for ROWNUM.
While you can certainly use ROWNUM, you should note that it is executed BEFORE any ORDER BY statement, thus producing wrong results in that case. The solution in this case is as simple as:
SELECT *
FROM (SELECT *
FROM myTable
ORDER BY myColumn)
WHERE ROWNUM = 1;
By encapsulating your query with the ORDER BY clause inside another query using ROWNUM, you now effectively order the dataset before extracting the TOP row. You may obviously use it as TOP N by setting another value for ROWNUM.
25/02/2013
Oracle delete purge object
In Oracle, when you delete a table with the DROP TABLE statement (or another object for that matter), it's not definitevly deleted as it goes instead into the recycle bin. This can be good as you may recover your objects with the FLASHBACK command but can also cause issues if other objects depend on or refer to the deleted one.
To permanently delete the object, you can either disable the recycle bin or use the PURGE command along with you DROP statement.
To permanently delete the object, you can either disable the recycle bin or use the PURGE command along with you DROP statement.
Oracle test if nested table has elements
If you're using nested tables in Oracle and need to check wether they're empty or not, you can use the IS EMPTY statement:
SELECT nt.*
FROM myTable t, TABLE(t.myNested) nt
WHERE t.myNested IS [NOT] EMPTY;
SELECT nt.*
FROM myTable t, TABLE(t.myNested) nt
WHERE t.myNested IS [NOT] EMPTY;
Oracle get nested table element value without column name
In Oracle, if you have a nested table with a single unnamed column (usually named simply "element"), you can access its value by using the COLUMN_VALUE keyword:
SELECT nt.COLUMN_VALUE
FROM myTable t, TABLE(t.myNested) nt;
SELECT nt.COLUMN_VALUE
FROM myTable t, TABLE(t.myNested) nt;
07/02/2013
Oracle get MAX or MIN between multiple values
Windows 7 enable Telnet client
So, you're happily using Windows 7 when suddenly you feel the urge to Telnet but your console says: 'telnet' is not recognized as an internal or external command, operable program or batch file.
If you get this error, you must add the Telnet client to your Windows installation. Go to Control panel->Programs and features->Turn Windows features on or off then scroll down to "Telnet Client", check it and press OK.
Now wait until installation is complete and you're done!
If you get this error, you must add the Telnet client to your Windows installation. Go to Control panel->Programs and features->Turn Windows features on or off then scroll down to "Telnet Client", check it and press OK.
Now wait until installation is complete and you're done!
28/01/2013
Opera quick proxy toggle button
If you use Opera and need to easily and quickly toggle on or off your proxy setting, much like Firefox's QuickProxy add-on, you may try this. Point your browser to this URL:
Then accept the installation and drag the button where you want. Clicking on it will toggle the proxy on and off and a long-click will bring you to the proxy configuration menu.
opera:/button/Enable%20proxy%20servers,,,%22Disable%20Proxy%22,Expand%20Enabled%20%3E%20Disable%20proxy%20servers,,,%22Enable%20Proxy%22,Expand%20Disabled%20+%20Show%20preferences,22,Proxy%20servers
Then accept the installation and drag the button where you want. Clicking on it will toggle the proxy on and off and a long-click will bring you to the proxy configuration menu.
25/01/2013
Liferay cannot access login page
If you tinker with Liferay's page permissions, you may accidentally edit those associated with the "Welcome" page, effectively preventing ANYONE from ever logging in again.
To regain access you may try this (replace VERSION with your own):
NOTE: With this, you are enabling the auto-login feature by ANY means on the portal, so remember to remove that line after you've successfully restored your situation.
Replacing LIFERAY_IP with your portal's IP (could be localhost), LIFERAY_PORT with your portal's port (could be 8080), ADMIN_USERNAME with the portal's admin username (could be test@liferay.com or simply test) and ADMIN_PASSWORD with the portal's admin password (could be test). EG:
http://localhost:8080?parameterAutoLoginLogin=test¶meterAutoLoginPassword=test
NOTE: You may input ANY user/password there, not necessarily the admin's ones, but that user must have the ability to edit the "Welcome" page's permissions, else he'll now be able to login albeit without the means to solve the issue.
To regain access you may try this (replace VERSION with your own):
- stop Liferay (stop Tomcat), on Linux you need to run: /liferay/apache-tomcat-VERSION/bin/shutdown.sh - you may need to be a superuser to do this. On Windows it's the same path but you should use shutdown.bat instead
- edit the portal-ext.properties file. You can find it under /liferay/apache-tomcat-VERSION/webapps/ROOT/WEB-INF/classes - and add this line:
auto.login.hooks=com.liferay.portal.security.auth.CASAutoLogin,com.liferay.portal.security.auth.FacebookAutoLogin,com.liferay.portal.security.auth.NtlmAutoLogin,com.liferay.portal.security.auth.OpenIdAutoLogin,com.liferay.portal.security.auth.OpenSSOAutoLogin,com.liferay.portal.security.auth.RememberMeAutoLogin,com.liferay.portal.security.auth.SiteMinderAutoLogin,com.liferay.portal.security.auth.ParameterAutoLogin
NOTE: With this, you are enabling the auto-login feature by ANY means on the portal, so remember to remove that line after you've successfully restored your situation.
- restart Liferay. It's the same as stopping but you should use startup instead of shutdown
- open any browser and point to:
Replacing LIFERAY_IP with your portal's IP (could be localhost), LIFERAY_PORT with your portal's port (could be 8080), ADMIN_USERNAME with the portal's admin username (could be test@liferay.com or simply test) and ADMIN_PASSWORD with the portal's admin password (could be test). EG:
http://localhost:8080?parameterAutoLoginLogin=test¶meterAutoLoginPassword=test
NOTE: You may input ANY user/password there, not necessarily the admin's ones, but that user must have the ability to edit the "Welcome" page's permissions, else he'll now be able to login albeit without the means to solve the issue.
23/01/2013
MySQL database import and export
Here's a run-down of some useful command line commands to manage the import/export operations of your MySQL database using mysqldump and mysql.
Note that the export operations will lock the database down, preventing any non-read operations to be executed on it along with slowing it down significantly.
Note that the export operations will lock the database down, preventing any non-read operations to be executed on it along with slowing it down significantly.
16/01/2013
[MySQL] Access denied for user ''@localhost
I had a MySQL database on CentOS 5 with user root and no password. I still don't know why, but he no longer had ALL rights on ANY schema, effectively rendering him powerless; plus after logging in:
mysql -h localhost -u root;
a simple
SELECT current_user();
would show '@localhost' and
show databases;
would show only show the information_schema and test schema.
There was no way I could get back my rights, even starting mysql with the --skip-grant-tables option held no positive results and the FAQ on MySQL's site did not cover my case. I kept getting the error: ERROR 1044 (42000): Access denied for user ''@'localhost' to database XXX
Additionally I could not select any database:
use XYZ;
Since I was told I did not have the rights on it. I couldn't even set or change the password neither from shell with:
mysqladmin -u root password 'NEW_PASSWORD'
to set or
mysqladmin -u root -p'OLD_PASSWORD' password 'NEW_PASSWORD'
to change, nor from the mysql prompt:
mysql -u root -p mysql
then:
UPDATE user SET password=PASSWORD('NEW_PASSWORD') where User='root';
FLUSH PRIVILEGES;
mysql -h localhost -u root;
a simple
SELECT current_user();
would show '@localhost' and
show databases;
would show only show the information_schema and test schema.
There was no way I could get back my rights, even starting mysql with the --skip-grant-tables option held no positive results and the FAQ on MySQL's site did not cover my case. I kept getting the error: ERROR 1044 (42000): Access denied for user ''@'localhost' to database XXX
Additionally I could not select any database:
use XYZ;
Since I was told I did not have the rights on it. I couldn't even set or change the password neither from shell with:
mysqladmin -u root password 'NEW_PASSWORD'
to set or
mysqladmin -u root -p'OLD_PASSWORD' password 'NEW_PASSWORD'
to change, nor from the mysql prompt:
mysql -u root -p mysql
then:
UPDATE user SET password=PASSWORD('NEW_PASSWORD') where User='root';
FLUSH PRIVILEGES;
07/01/2013
Oracle create database link and error ORA-12514
Creating a database link in Oracle is pretty straightforward and both the official documentation and Toad's Knowledge Xpert well detail the steps.
What might happen afterwards however, is that you successfully test your DB link with a client like Toad or SQLPlus, but get the ORA-12514 "TNS: listener does not currently know of service" error when using it from other clients.
This lengthy post goes well deep into how to troubleshoot errors like it, but the most basic thing you should always check first is "Are my TNSNAMES.ORA files correctly configured on both the local and remote DB?"
You may find that the reason why the client you tested the DB link with worked but the link does not work from another source instead, is that your client uses its own copy of the TNSNAMES file while the one read by your DB server is not correctly updated.
What might happen afterwards however, is that you successfully test your DB link with a client like Toad or SQLPlus, but get the ORA-12514 "TNS: listener does not currently know of service" error when using it from other clients.
This lengthy post goes well deep into how to troubleshoot errors like it, but the most basic thing you should always check first is "Are my TNSNAMES.ORA files correctly configured on both the local and remote DB?"
You may find that the reason why the client you tested the DB link with worked but the link does not work from another source instead, is that your client uses its own copy of the TNSNAMES file while the one read by your DB server is not correctly updated.
Subscribe to:
Posts (Atom)