Commit cd527ef4 authored by Abdul R. Wahid's avatar Abdul R. Wahid
Browse files

Merge branch 'develop' into 'master'

Develop

See merge request riset/onest!3
parents efadc6d3 11887bc8
NODE_ENV=
HOST=
PORT=
NAME=
DESC=
VERSION=
JWT_SECRET=
JWT_EXPIRE=
JWT_PRIVATE_KEY=
JWT_PUBLIC_KEY=
JWT_JWKS=
JWT_JWKS_KEYS=
DB_MYSQL_HOST=
DB_MYSQL_PORT=
DB_MYSQL_USER=
DB_MYSQL_PASS=
DB_MYSQL_NAME=
DB_PG_HOST=
DB_PG_PORT=
DB_PG_USER=
DB_PG_PASS=
DB_PG_NAME=
DB_MONGO_URL=
\ No newline at end of file
......@@ -2,6 +2,8 @@
config/*
!config/config*
!config/sample*
!config/keys/jwt/.gitkeep
.env
# compiled output
/dist
......@@ -38,3 +40,12 @@ lerna-debug.log*
!.vscode/launch.json
!.vscode/extensions.json
package-lock.json
# Uploaded & Generated File
storage/upload/*
!storage/upload/.gitkeep
storage/playground/jwt/*
!storage/playground/jwt/.gitkeep
# Documentation
documentation
......@@ -34,6 +34,95 @@
$ npm install
```
## Setup
### Initial Setup
#### App Configuration
Copy configurations from config/sample* to config/{environment}*.
You can save your configuration to shared/secured storage or commiting them.
You need to unignore environment configurations from .gitignore to commiting them.
```bash
# Example
!config/development*
```
#### .env Configuration
Input required configurations that need be overrided with environment variables (See .env.sample / config/config.schema.ts)
#### JWT Configuration
##### Using JWT Keys
Copy generated jwt keys from your shared/secured storage (mostly for microservice architecture)
OR
Generate new jwt keys for your application (mostly for service oriented architecture)
```bash
# generate jwt keys
$ node bin/jwt-tool/jwt-key-gen.js
```
You can save your jwt keys to shared/secured storage or commiting them.
You need to unignore jwt keys from .gitignore to commiting them.
```bash
# Example
!config/keys*
```
##### Using JWT Secret
Fill the secret in configuration file or .env
#### Database Migration
```bash
# = Development Environment =
# Migrate database to latest version
$ npm run db:dev migrate:latest
# Apply seeders to fill database (optional, recommended)
$ npm run db:dev seed:run
```
### Development / Deployment Setup
#### App Configuration
Use Commited configurations
OR
Copy configurations from your shared/secured storage
OR
Copy configurations from config/sample* to config/{environment}*
#### .env Configuration
Input required configurations that need be overrided with environment variables (See .env.sample / config/config.schema.ts)
#### JWT Configuration
##### Using JWT Keys
Use Commited configurations
OR
Copy configurations from your shared/secured storage
##### Using JWT Secret
Fill the secret in configuration file or .env
#### Database Migration
```bash
# = Development Environment =
# Migrate database to latest version
$ npm run db:dev migrate:latest
# WARNING!!! NEVER DO IN PIPELINE!!!
# Apply seeders to fill database (Run Manual - Optional on First Time Only)
$ npm run db:dev seed:run
# = Production Environment =
# Migrate database to latest version
$ npm run db:prod migrate:latest
# WARNING!!! NEVER DO IN PIPELINE!!!
# Apply seeders to fill database (Run Manual - Optional on First Time Only)
$ npm run db:prod seed:run
# = Other Environment =
# Migrate database to latest version
$ npm run db migrate:latest -- --env staging
# WARNING!!! NEVER DO IN PIPELINE!!!
# Apply seeders to fill database (Run Manual - Optional on First Time Only)
$ npm run db seed:run -- --env staging
```
## Running the app
```bash
......@@ -60,6 +149,118 @@ $ npm run test:e2e
$ npm run test:cov
```
## Directory Structure
```txt
-- root
+-- assets**
+-- bin*
+-- config**
+-- data*
+-- dist*
+-- dump***
+-- node_modules*
+-- src
+-- common
+-- core
+-- database
+-- module
+-- utils
+-- storage**
+-- test
.gitignore
.prettierrc
.env.sample
.env**
knexfile.ts*
nest-cli.json*
package-lock.json*
package.json*
README.md
tsconfig.build.json*
tsconfig.json*
tslint.json
Notes:
: Ignored on deployment
* : Must copied as is on deployment
** : Can be copied as is or Mounted from another storage on deployment
*** : Deprecated, moved or removed in the future
```
### The root directory
#### The assets directory
Contains assets and resources that will be loaded or compiled in run-time.
#### The bin directory
Contains custom script that usefull for development / deployment.
#### The config directory
Contains configuration files.
#### The data directory
Contains file to supporting database / provider operations, for example: migrations; seeds; dumps; etc.
#### The dist directory
Contains builded application code.
#### The dump directory
Contains database dumps before boilerplate uses database migration.
#### The node_modules directory
Contains Node.js dependency modules.
#### The src directory
Contains main application code.
#### The storage directory
Contains uploaded files, logs, caches, and other generated files.
#### The test directory
Contains main application test code.
### The src directory
#### The src/common directory
Contains application classes/files that might commonly be used by other modules or globally by the application.
That files includes: decorator, filter, interceptor, exception, guard, helper, interface, middleware, pipe, serializer
#### The src/core directory
Contains boilerplate's core modules (especially for auth, user management, and simple media management).
#### The src/database directory
Contains bolerplate's database provider modules.
#### The src/module directory
Contains main applications modules.
#### The src/utils directory
Contains boilerplate's and application shared utiliy modules, that can be used accross applications.
## Tips
### API Versioning
Before version 8.0 Nest.js doesn't support api versioning by default. We can create versioned modules directory with following styles:
```bash
# style 1
+-- module
+-- ...modules...
+-- module2
+-- ...modules...
# style 2
+-- module
+-- ...modules...
+-- v2
+-- ...modules...
# style 3
+-- module
+-- ...core module extends (if any)...
+-- v1
+-- ...modules...
+-- v2
+-- ...modules...
```
## Support
Nest is an MIT-licensed open source project. It can grow thanks to the sponsors and support by the amazing backers. If you'd like to join them, please [read more here](https://docs.nestjs.com/support).
......
const BASEDIR = './config/keys/jwt/'
const TEMPDIR = './storage/playground/jwt/'
module.exports = {
BASEDIR,
TEMPDIR
}
const fs = require('fs')
const { BASEDIR } = require('./constant')
const deleteKey = (name, data) => {
const file = BASEDIR + name
if (fs.existsSync(file)) {
try {
fs.unlinkSync(file)
console.log(`Successfully Deleted JWT Key (${file})`)
} catch (error) {
console.log(`Failed to Delete JWT Key (${file})`)
}
return
}
console.log(`No JWT Key (${file}) to be deleted!`)
}
deleteKey('jwt.jwkspub.json')
deleteKey('jwt.jwks.json')
deleteKey('jwt.public.pem')
deleteKey('jwt.private.pem')
const fs = require('fs')
const jose = require('jose')
const { BASEDIR } = require('./constant')
// Guide Example: https://ichi.pro/id/jwk-dan-node-jose-161026790103554
//
// const nodeJose = require('node-jose')
// const keyStore = nodeJose.JWK.createKeyStore()
// keyStore.generate('RSA', 2048, { alg: 'RS256', use: 'sig' }).then(result => {
// fs.writeFileSync(
// BASEDIR + 'jwt.keys.json',
// JSON.stringify(keyStore.toJSON(true), null, ' '),
// )
// })
const writeKey = (name, data) => {
const file = BASEDIR + name
if (!fs.existsSync(BASEDIR)) {
fs.mkdirSync(BASEDIR, { recursive: true })
}
if (fs.existsSync(file)) {
return console.log(`JWT Key (${name}) Exist! Please delete them first using "bin/jwt-tool/jwt-key-del.js" before regenerate.`)
}
fs.writeFileSync(file, data)
console.log(`Successfully Generated JWT Key (${file})`)
}
const keyStore2 = new jose.JWKS.KeyStore()
jose.JWK.generate('RSA', 2048, { alg: 'RS256', use: 'sig' }).then(result => {
keyStore2.add(result)
writeKey('jwt.jwkspub.json', JSON.stringify(keyStore2.toJWKS(), null, ' '))
writeKey('jwt.jwks.json', JSON.stringify(keyStore2.toJWKS(true), null, ' '))
writeKey('jwt.public.pem', result.toPEM())
writeKey('jwt.private.pem', result.toPEM(true))
})
const fs = require('fs')
const jwt = require('jsonwebtoken')
const { BASEDIR, TEMPDIR } = require('./constant')
const loadToken = (name = 'input.txt') => {
const file = TEMPDIR + name
if (fs.existsSync(file)) {
console.log(`Using token from (${file})`)
return fs.readFileSync(file).toString()
}
if (name !== 'output.txt') {
return loadToken('output.txt')
}
console.log('No token found!')
return null
}
const loadKey = (name) => {
const file = BASEDIR + name
if (fs.existsSync(file)) {
console.log(`Using key from (${file})`)
return fs.readFileSync(file)
}
console.log(`No JWT Key (${file}) found!`)
return null
}
const TOKEN = loadToken()
const publicKey = loadKey('jwt.public.pem')
const decoded = jwt.verify(TOKEN, publicKey)
console.log(decoded)
const fs = require('fs')
const jose = require('jose')
const { BASEDIR, TEMPDIR } = require('./constant')
const writeToken = (name, data) => {
const file = TEMPDIR + name
if (!fs.existsSync(TEMPDIR)) {
fs.mkdirSync(TEMPDIR, { recursive: true })
}
fs.writeFileSync(file, data)
console.log(`Generated token saved to: (${file})`)
}
const jwks = fs.readFileSync(BASEDIR + 'jwt.jwks.json')
const PAYLOAD = {
sub: '3@user',
user_id: 3,
user_group: 3,
user_org: 3,
user_verified: 1,
groups: [
{
org: 3,
group: 3,
code: 'customer',
},
],
roles: ['customer'],
entity: 'user',
}
PAYLOAD.iat = Math.floor(Date.now() / 1000)
PAYLOAD.exp = Math.floor((Date.now() + 1000 * 60 * 60 * 24) / 1000)
// Guide Example : https://ichi.pro/id/jwk-dan-node-nodeJose-161026790103554
// const nodeJose = require('node-jose')
// const keyStore = await nodeJose.JWK.asKeyStore(jwks.toString())
// const [ key ] = keyStore.all({ use: 'sig' })
// const opt = { compact: true, jwk: key, fields: { typ: 'jwt' } }
// const payload = JSON.stringify({
// exp: Math.floor((Date.now() + ms('1d')) / 1000),
// iat: Math.floor(Date.now() / 1000),
// sub: 'test',
// })
// const token = await nodeJose.JWS.createSign(opt, key)
// .update(payload)
// .final()
const keyStoreLoader = jose.JWKS.asKeyStore(JSON.parse(jwks.toString()))
const [firstKey] = keyStoreLoader.all({ use: 'sig' })
const tokenWithJWS = jose.JWS.sign(PAYLOAD, firstKey, { typ: 'jwt', kid: firstKey.kid })
const tokenWithJWT = jose.JWT.sign(PAYLOAD, firstKey, { header: { typ: 'jwt' } })
console.log({
tokenWithJWS,
isSame: tokenWithJWS === tokenWithJWT,
tokenWithJWT
})
writeToken('output.txt', tokenWithJWT)
import * as path from 'path'
import * as fs from 'fs'
import * as _ from 'lodash'
import * as convict from 'convict'
import * as pkg from '../package.json'
import * as dotenv from 'dotenv'
// Load & Sync .env
dotenv.config()
// Define a schema
import configSchemaExtend from '@config/config.schema'
const configSchemaBase = {
env: {
doc: "The application environment.",
format: ["production", "development", "staging", "test"],
default: "development",
env: "NODE_ENV",
arg: 'env'
},
host: {
doc: "The host to bind.",
format: String,
default: "127.0.0.1",
env: "HOST",
arg: 'host'
},
port: {
doc: "The port to bind.",
format: "port",
default: 3000,
env: "PORT",
arg: "port"
}
env: {
doc: 'The application environment.',
format: ['production', 'development', 'staging', 'test'],
default: 'development',
env: 'NODE_ENV',
arg: 'env'
},
host: {
doc: 'The host to bind.',
format: String,
default: '127.0.0.1',
env: 'HOST',
arg: 'host'
},
port: {
doc: 'The port to bind.',
format: 'port',
default: 3000,
env: 'PORT',
arg: 'port'
},
name: {
doc: 'The app name.',
format: String,
default: pkg.name,
env: 'NAME',
arg: 'name'
},
desc: {
doc: 'The app desc.',
format: String,
default: pkg.description,
env: 'DESC',
arg: 'desc'
},
version: {
doc: 'The app version.',
format: String,
default: pkg.version,
env: 'VERSION',
arg: 'version'
}
}
const configSchema = _.mergeWith(configSchemaBase, configSchemaExtend, (objValue, srcValue) => {
if (_.isArray(objValue)) {
return objValue.concat(srcValue);
}
if (_.isArray(objValue)) {
return objValue.concat(srcValue)
}
}
)
// Init config
const config = convict(configSchema)
// Sync env
let env = process.env.NODE_ENV = config.get('env')
const env = process.env.NODE_ENV = config.get('env')
// Load environment dependent configuration
const data = require(`@config/${env}`)
config.load(data.default)
if (fs.existsSync(`./config/${env}.ts`)) {
const data = require(`@config/${env}`)
config.load(data.default)
} else {
console.log(`[Config] - ${(new Date()).toString()} [Loader] Not load ${env} environment configuration file`)
}
// Perform validation
config.validate({ allowed: 'strict' })
......
const schema = {
authlog: {
active: {
doc: "Service Active",
format: Boolean,
default: true
}
authlog: {
active: {
doc: 'Service Active',
format: Boolean,
default: true
}
},
pagination: {
page: {
doc: 'Default Pagination Page',
format: Number,
default: 1
},
view: {
doc: 'Default Pagination View',
format: Number,
default: 10
}
},
auth: {
register: {
admin: {
format: Boolean,
default: false
},
adminGroup: {
format: Number,
default: 2
},
adminOrg: {
format: Number,
default: 2
},
user: {
format: Boolean,
default: true
},
userGroup: {
format: Number,
default: 3
},
userOrg: {
format: Number,
default: 3
}
},
password: {
hash: {
format: ['md5', 'bcrypt', 'plain'],
default: 'md5'
},
compatibility: {
format: Boolean,
default: false
}
},
verification: {
expiry: {
format: Number,
default: 2
},
tokenPrivateLength: {
format: Number,
default: 6
}
},
login: {
unverified: {
format: Boolean,
default: true
}
},
forgot: {
expiry: {
format: Number,
default: 5
}
}
},
profile: {
change: {