[Google Sheets] Import data from Investing.com

DISCLAIMER: This is not a stock picking recommendation, the tickers shown are used as example and not provided as financial or investment advice.

DISCLAIMER2: Web scraping can go against usage policies so read them, understand them and abide by them. Also web scraping is notoriously flaky as small changes in the retrieved page can affect the location of the desired element(s).

We have already seen how to import data from Yahoo Finance and Coingecko, now we try to add Investing.com as source which unfortunately does not provide API, so we have to do some HTML scraping instead.

We would like to retrieve the current price of an ETF called JPNA. By looking at that page we can luckily identify a specific locator, which then allows us to do a couple string manipulation operations before getting the desired value:

function getInvestingData(path) {
  var result = UrlFetchApp.fetch("https://www.investing.com/etfs/" + path.toLowerCase());
  var response = result.getContentText();
  var start = response.indexOf(' data-test="instrument-price-last">') + 35;
  var end = response.indexOf('</div>', start)
  var out = response.substring(start, end);
  var strip = out.replace(",", "");
  return Number(strip);
}

[Google Sheets] Import crypto data from Coingecko

DISCLAIMER: I do not recommend investing in crypto, all tokens you will see are used as sample and not provided as financial or investment advice.

We've already seen how to add custom functions to our Google Sheets projects, importing Yahoo Finance data. Now we expand this to import cryptocurrency data from Coingecko.

I could have picked any of the million sources, but this one has a simple free plan which for light usage works well.

After registering and getting your API key, RTFM to find that they provide a lot of stuff, what we care about is the pricing data in this example, you will need to find the ID of the currency you want (NOT the token) by querying the list and then you can get the data you want by calling (add your API key at the end):

"https://api.coingecko.com/api/v3/simple/price?ids=" + token + "&vs_currencies=usd&x_cg_demo_api_key="

You can put this in a Google Sheet function as well (you can extend input parameters to get quote in different currencies as well):

function getCoingeckoData(path) {
  var result = UrlFetchApp.fetch("https://api.coingecko.com/api/v3/simple/price?ids=" + path + "&vs_currencies=usd&x_cg_demo_api_key=YOUR_API_KEY");
  var response = result.getContentText();
  var json = JSON.parse(response);
  return json[path]['usd'];
}

[Java] Get type of elements in collection using reflection

Java type erasure logic means that at runtime some information is removed/replaced from generic declarations, which poses a minor challenge when trying to identify the actual parameter types using reflection at runtime.

Here is a way:

import java.lang.reflect.Field;
import java.lang.reflect.ParameterizedType;


//maybe you are looping over fields or have a field of type Field f
ParameterizedType collectionType = (ParameterizedType) f.getGenericType();
Class<?> actualClassOfElementInCollection = (Class<?>) collectionType.getActualTypeArguments()[0];

[Java] Load entity with lazy children collections

When using JPA and lazy collection loading, at the time the parent entity is retrieved from DB, the lazy children are NOT also loaded in memory, instead, a proxy reference is added, which is used to retrieve the data if that particular field is ever accessed.

In some scenarios you might want to load the whole entity, including the lazy children in memory instead (eg you want to clone/serialize it, whatever).

If you want to load ONE child together with the parent, a JOIN FETCH clause would do the trick: 

SELECT p

FROM Parent p

LEFT JOIN FETCH p.childField c

WHERE p.id = :id

But if you try to load more than one child at the same time, you will get a MultipleBagFetchException. A workaround is to call the load with JOIN FETCH for all entities you need sequentially, for example:

private Parent loadChildViaQuery(ID id, String childClause) {
  return entityManager
    .createQuery(
      "select p " +
      "from Parent p " +
      "left join fetch " +
      childClause +
      " where p.id = :id",
      Parent.class
    )
    .setParameter("id", id)
    .getSingleResult();
}

Where childClause input is the join statement you need, for example:

"p.childField c"

Also remember that if the parent entity is not found, the operation would throw a NoResultException, while if the child is not found, no exception is raised, the child is simply null/empty.

[Java] Using annotation processing (and validating it) to execute logic at runtime

Sample project showcasing how to use annotations to perform runtime logic to log changes in object values.

Remember it is a SAMPLE, so obviously (lazy me) most null-safe checks and so are not included and obviously some logic is a showcase, should be replaced with a real business scenario to implement.

In this SAMPLE, we tag fields to be included in a diff logic to then print to output when those fields values change by comparing two instances of the same object.

Logic can obviously be made much more complex including collections and maps and whatnot (Java Generics are your friends there).

Also worth mentioning JaVers can do most of it for you, unless you have fancy business requirements that force you to write custom code..

Key points:

- How to create an annotation

- How to create an annotation processor to validate the annotation parameters at compile time

- How to register an annotation processor

- How to configure a multi-module Maven project to use a custom annotation processor (also, in the pom of the root project ensure the module containing the processor is built BEFORE everything else)

- How to test an annotation processor by generating classes at runtime and trigger compilation tasks agains them. Includes verifying compilation warnings are properly triggered as expected.

- How to use reflection to get fields annotated with a given annotation (and then execute whatever logic on them)

- How to use reflection to invoke methods (including static methods)

The full project is available on my GitHub repo with commented code: https://github.com/steghio/diff-annotation-processing

[Java] Serialize POJO to XML according to XSD

If you generate a class from an XSD schema, it will come with the necessary annotations to serialize it to an XML String.

You can therefore easily convert it with:

import jakarta.xml.bind.JAXBContext;
import jakarta.xml.bind.Marshaller;
import java.io.StringWriter;

/**
 * Provides utility methods for serialization scenarios
 */
public class SerializationUtils {

  /**
   * Serialize the given object to XML String using JAXBContext
   * It will set the output to be pretty printed
   * It relies on the object annotations to correctly place and annotate all fields
   * @param object
   * @return the string representation of this object as XML
   * @param <T>
   */
  public static <T> String serializeXml(T object) {
    try {
      JAXBContext jc = JAXBContext.newInstance(object.getClass());

      Marshaller marshaller = jc.createMarshaller();
      marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true);
      //to completely remove the xml preamble `<?xml version="1.0" encoding="UTF-8" standalone="yes"?>` add this line:
      //marshaller.setProperty(Marshaller.JAXB_FRAGMENT, true);

      //marshaller cannot output to string directly
      StringWriter sw = new StringWriter();

      marshaller.marshal(object, sw);

      return sw.toString();
    } catch (Exception e) {
      throw new RuntimeException("Failed to convert payload to xml. ", e);
    }
  }
}

[Java] Generate POJO from XSD in Maven

Assuming you have a nice correct XSD file with proper namespace references and all, then you could convert it to a POJO (or more) using jaxb-maven-plugin

There are multiple plugins that would achieve the same result and multiple versions of this plugin even, so searching on the web can be confusing. In year 2024, this works simply with adding a plugin in the POM:

<plugin>
  <groupId>org.jvnet.jaxb</groupId>
  <artifactId>jaxb-maven-plugin</artifactId>
  <version>4.0.8</version>
  <executions>
    <execution>
      <id>NAME_FOR_THIS_RUN</id>
      <goals>
        <goal>generate</goal>
      </goals>
      <configuration>
        <schemaDirectory>src/main/resources/FOLDER/USE_CASE</schemaDirectory> <!-- here will be the XSD -->
      </configuration>
    </execution>
  </executions>
</plugin>