In this tutorial, we’ll be discussing Augmented Reality and implement a hello world sample in our android application using Sceneform.
Table of Contents
Augmented Reality
Augmented Reality is gaining more popularity day by day. It’s used to show you virtual 3d models/graphics in a real world through your mobile’s camera. It’s useful in demonstrating simulated stuff in the real world such as paints/decoration you’d like to see on your wall.
Augmented Reality lies somewhere between Virtual Reality and Real environment.
Android ARCore
Google has come up with ARCore library to enhance AR development.
ARCore does motion tracking in the real world through the camera to create paths on planes/surfaces over which we can place our 3D Models and graphics.
SceneForm is a 3D framework that’s come up recently and is a better alternative to OpenGL. OpenGL looks intimidating to code.
SceneForm allows us to quickly render 3d objects without the need to learn graphics or OpenGL.
We can download the Google Sceneform Tools plugin in our Android Studio to view and render the 3d models.
You can go to https://poly.google.com/ and download a sample model. Don’t forget to credit the creator!
Typically, the OBJ and GLTX formats are used for rendering augmented images.
Now let’s build our first AR application where we’ll use the above 3D model as our AR image.
Augmented Reality Android Example Project Structure
In the above project, we’ve created a sampledata
directory, where we add the obj and mtl file downloaded earlier.
Add the following dependency in your project’s build.gradle:
classpath 'com.google.ar.sceneform:plugin:1.0.1'
Add the following in your app’s build.gradle :
implementation 'com.google.ar.sceneform.ux:sceneform-ux:1.0.0'
You need to set the ar plugin in your app’s Gradle file as well.
Add the following below the dependencies:
apply plugin: 'com.google.ar.sceneform.plugin'
We have used the following image, you can download it from this link and put it in the sampledata directory:
Now right click on the obj file and choose import sceneform assets:
This would automatically create the Renderable AR sceneform assets. It would automatically add the following lines at the end of the app’s build.gradle:
sceneform.asset('sampledata/Coffee Cup_final.obj',
'default',
'sampledata/Coffee Cup_final.sfa',
'src/main/assets/Coffee Cup_final')
The build.gradle of the app looks like this finally:
Android ARCore Sceneform requires Java 8 or higher version.
The sja and sjb are the Sceneform Asset Description
and Sceneform Binary
files.
The sjb file is visible in the 3D viewer. It is shipped with the APK.
The sja file is used to set properties for the sjb file.
scale
of the model in the sja file.Android Augmented Reality Example Code
To configure ARCore in your application add the following permission and metadata to your AndroidManifest.xml file.
<uses-permission android:name="android.permission.CAMERA" />
<uses-feature
android:name="android.hardware.camera.ar"
android:required="true" />
metadata goes inside the application tag.
<meta-data
android:name="com.google.ar.core"
android:value="required" />
The code for the activity_main.xml class is given below:
<?xml version="1.0" encoding="utf-8"?>
<android.support.design.widget.CoordinatorLayout xmlns:android="https://schemas.android.com/apk/res/android"
xmlns:app="https://schemas.android.com/apk/res-auto"
xmlns:tools="https://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
tools:context=".MainActivity">
<android.support.design.widget.AppBarLayout
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:theme="@style/AppTheme.AppBarOverlay">
<android.support.v7.widget.Toolbar
android:id="@+id/toolbar"
android:layout_width="match_parent"
android:layout_height="?android:attr/actionBarSize"
android:background="?android:attr/colorPrimary"
app:popupTheme="@style/AppTheme.PopupOverlay" />
</android.support.design.widget.AppBarLayout>
<include layout="@layout/content_main" />
<android.support.design.widget.FloatingActionButton
android:id="@+id/fab"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_gravity="bottom|end"
android:layout_margin="@dimen/fab_margin"
app:srcCompat="@android:drawable/ic_input_add" />
</android.support.design.widget.CoordinatorLayout>
The code for the content_main.xml
is given below:
<?xml version="1.0" encoding="utf-8"?>
<android.support.constraint.ConstraintLayout xmlns:android="https://schemas.android.com/apk/res/android"
xmlns:app="https://schemas.android.com/apk/res-auto"
xmlns:tools="https://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
app:layout_behavior="@string/appbar_scrolling_view_behavior"
tools:context=".MainActivity"
tools:showIn="@layout/activity_main">
<fragment
android:id="@+id/sceneForm"
android:name="com.google.ar.sceneform.ux.ArFragment"
android:layout_width="match_parent"
android:layout_height="match_parent"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toTopOf="parent" />
</android.support.constraint.ConstraintLayout>
We’ve set the fragment as ArFragment. The ArFragment
checks if the phone is compatible with ARCore.
Also, it checks if the camera permission is granted. If it isn’t, it’ll ask for it automatically.
Also, it’ll ask you to download the ARCore by Google Application.
The code for MainActivity.java class is given below:
package com.journaldev.androidarsceneform;
import android.graphics.Point;
import android.net.Uri;
import android.os.Bundle;
import android.support.annotation.Nullable;
import android.support.design.widget.FloatingActionButton;
import android.support.v7.app.AlertDialog;
import android.support.v7.app.AppCompatActivity;
import android.support.v7.widget.Toolbar;
import android.view.View;
import android.view.Menu;
import android.view.MenuItem;
import com.google.ar.core.Anchor;
import com.google.ar.core.Frame;
import com.google.ar.core.HitResult;
import com.google.ar.core.Plane;
import com.google.ar.core.Trackable;
import com.google.ar.sceneform.AnchorNode;
import com.google.ar.sceneform.rendering.ModelRenderable;
import com.google.ar.sceneform.ux.ArFragment;
import com.google.ar.sceneform.ux.TransformableNode;
import java.util.List;
import java.util.function.Consumer;
import java.util.function.Function;
public class MainActivity extends AppCompatActivity {
ArFragment arFragment;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
Toolbar toolbar = (Toolbar) findViewById(R.id.toolbar);
setSupportActionBar(toolbar);
FloatingActionButton fab = (FloatingActionButton) findViewById(R.id.fab);
fab.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View view) {
addObject(Uri.parse("Coffee Cup_Final.sfb"));
}
});
arFragment = (ArFragment) getSupportFragmentManager().findFragmentById(R.id.sceneForm);
}
@Override
public boolean onCreateOptionsMenu(Menu menu) {
// Inflate the menu; this adds items to the action bar if it is present.
getMenuInflater().inflate(R.menu.menu_main, menu);
return true;
}
@Override
public boolean onOptionsItemSelected(MenuItem item) {
// Handle action bar item clicks here. The action bar will
// automatically handle clicks on the Home/Up button, so long
// as you specify a parent activity in AndroidManifest.xml.
int id = item.getItemId();
//noinspection SimplifiableIfStatement
if (id == R.id.action_settings) {
return true;
}
return super.onOptionsItemSelected(item);
}
private void addObject(Uri parse) {
Frame frame = arFragment.getArSceneView().getArFrame();
Point point = getScreenCenter();
if (frame != null) {
List<HitResult> hits = frame.hitTest((float) point.x, (float) point.y);
for (int i = 0; i < hits.size(); i++) {
Trackable trackable = hits.get(i).getTrackable();
if (trackable instanceof Plane && ((Plane) trackable).isPoseInPolygon(hits.get(i).getHitPose())) {
placeObject(arFragment, hits.get(i).createAnchor(), parse);
}
}
}
}
private final void placeObject(final ArFragment fragment, final Anchor createAnchor, Uri model) {
ModelRenderable.builder().setSource(fragment.getContext(), model).build().thenAccept((new Consumer() {
public void accept(Object var1) {
this.accept((ModelRenderable) var1);
}
public final void accept(ModelRenderable it) {
if (it != null)
MainActivity.this.addNode(arFragment, createAnchor, it);
}
})).exceptionally((new Function() {
public Object apply(Object var1) {
return this.apply((Throwable) var1);
}
@Nullable
public final Void apply(Throwable it) {
AlertDialog.Builder builder = new AlertDialog.Builder(MainActivity.this);
builder.setMessage(it.getMessage()).setTitle("error!");
AlertDialog dialog = builder.create();
dialog.show();
return null;
}
}));
}
private void addNode(ArFragment fragment, Anchor createAnchor, ModelRenderable renderable) {
AnchorNode anchorNode = new AnchorNode(createAnchor);
TransformableNode transformableNode = new TransformableNode(fragment.getTransformationSystem());
transformableNode.setRenderable(renderable);
transformableNode.setParent(anchorNode);
fragment.getArSceneView().getScene().addChild(anchorNode);
transformableNode.select();
}
private Point getScreenCenter() {
View vw = findViewById(android.R.id.content);
return new Point(vw.getWidth() / 2, vw.getHeight() / 2);
}
}
The anchor is the position where the model or the Node is placed in the screen. It’s the center of the screen here.
In the addObject()
the ARFragment gets the hit points on the plane using motion tracking.
placeObject()
asynchronously sets the node to the center of the screen.
Following are the outputs when the application is launched:
In the final output, the cup is positioned on the plane which consists of white dots.
The above screenshot is from Samsung S8.
Now try changing the model scale in SFA file!
This brings an end to this tutorial. You can download the final AndroidArSceneform Project from the link below.
The size of the project is big, thanks to the OBJ file.
hello
I ‘m trying to clicked on th obj file but did not showing import sceneform assets
Hello, I would like to know if you could make an explanatory video on how to use images as markers for augmented reality in android studio, because I do not find much information on how to do it through android studio and if you can make the video using the latter version of android studio because it could not do it with the latest version.
hello sir, I want to know how to make AR measure application as it is in demand. kindly upload the tutorials for AR Measure application.
Getting the below error. 3D model is not showing
ARCore-LocationProvider: Exception during FLP api client construction: com.google.android.gms.common.api.GoogleApiClient
Hello sir … I would like to thank you for your all works you are amazing. I want to ask you one question please do a tutorial on Location-based Augmented Reality that mark the place
Thank you
hello,
i’m facing problem your code.android resource linking failed error is showing.
Hello!
Maybe you can still help me, if you see this at all.
What I wanted to do was , instead of using the button ,I wish to something when I click the 3d model itself.
When I try to do what you did to the button to a node I always get an error
I want to place an object without tapping on a particular location how to place ?
Coordinates!
hi,
i have seen your tutorials for how to use AR core but your code is not working properly for detect image with obj file
Hello Anupam Chugh,
I am newbie to ARCore, Sceneform and I’m looking for some tutorial or sample of targeting image with buttons (maybe buttons are floating just like in Sceneform Solar System Sample). I hope you can help me! Thanks for your great tutorial sir.
Not working
Can you elaborate on the error you’re getting?
hello sir i am facing error Coffee Cup_Final.sfb file not found , how to solve please guide me
i modified in assets the file to coffe.sfb, and it worked
Nice!
I ‘m trying to right click on th obj file but did not showing import sceneform assets.
I’m using Android Studio 4.1 .
There one more problem :
Plugin Error: Plugin “Google Sceneform Tools (Beta)” is incompatible (supported only in IntelliJ IDEA).
Thanks in Advance